CN1278533C - Handset capable of automatically recording characters and images, and method of recording and processing thereof - Google Patents

Abstract

The invention relates to a mobile phone capable of automatically inputting text and images and a method for inputting and processing the same, for example, realizing full-automatic identification, input and processing of business card information on the mobile phone. The mobile phone includes a camera, and its camera signal output end is connected with the central processing unit of the mobile phone. Under the control of the general control module, the camera driver module takes pictures of the words and images on the business card; the layout processing and enhancement module, the intelligent layout analysis module, the character recognition module and the post-processing module work sequentially to perform preprocessing and processing on the camera signal. Enhanced processing, segmenting each text area and graphic area, identifying and classifying the characters in the text area, and storing the classification results and graphic compression results in the database module. The data exchange module can realize the data exchange with the computer or with other mobile phones through the local network or the remote network connection; the display drive module can drive the display; and the associated application module can perform associated application on the selected business card information.

Description

Mobile phone capable of automatically inputting text and images and its input and processing method

technical field

The invention relates to a mobile communication product, more specifically to a mobile phone device capable of automatically entering text and image information and a method for automatically entering and processing them.

Background technique

In order to maintain the continuity of social and business communication activities, it is necessary to retain the corresponding information of contacts and their contact information. As a traditional carrier of contact information, business cards have been widely used for their advantages of portability, price and practicality. Widely used, but it has the disadvantages of not being easy to save, fast updating and difficult to find.

The phone book function is an important function in mobile phones at present. Users can use the phone book function to record contacts and their contact numbers anytime and anywhere, and use this information to communicate with contacts in real time and facilitate information updates.

With the increasing popularity of mobile phones and the increasingly powerful computing power of mobile phones, how to automatically and quickly input and manage business card information into mobile phones has become an important demand for business mobile phones.

There are roughly three ways to enter business card information in existing mobile phones:

The first is the manual input of the keyboard, which mainly uses the mobile phone keyboard to manually enter the contact’s business card information into the phone book of the mobile phone word by word; the second is the manual input of the handwriting method, which is configured on the mobile phone. Handwriting touch screen, input the business card information of the contact into the phone book of the mobile phone word by word through manual handwriting; the third is the way of connecting the computer to a scanner, the scanner scans the business card, inputs the scanned information into the computer, and then the computer sends the contact’s information to the computer. Enter the business card information into the phone book of the mobile phone.

The main disadvantages of the above three methods are:

The manual input method of the mobile phone keyboard is not only cumbersome but also slow to input word by word, especially when entering the Email and website information in the business card, it is necessary to switch between Chinese and English repeatedly, which actually causes obstacles to the use of the phone book function of the mobile phone. Also lack the means of inputting graphic information such as logo on the business card (as company graphic mark, product trade mark pattern etc.).

The handwriting input method using the handwriting touch screen also needs to be input character by word/word, and the input speed is relatively slow. Since the mobile phone needs to provide handwriting input function, and the handwriting touch screen hardware is added, it is not suitable for keyboard-type mobile phones. In addition, it also lacks support for business cards. The input means of graphic information such as logo.

The input method of using an external business card scanner connected to a computer, because a personal computer and a business card scanner need to be configured, has not only increased the cost but also is not portable, and its scanning, identification, and output modules are all implemented in a personal computer, so it cannot be solved. The problem of directly inputting the contact card information into the phone book of the mobile phone, and the computer-based business card scanning and recognition system still needs to store the image information of the entire business card, which requires high storage capacity.

At present, although mobile phone products with a camera function have come out, only taking pictures and displaying them on a display screen requires the help of a PC to integrate the information on the business card into the phone book.

Contents of the invention

The purpose of the present invention is to design a mobile phone that can automatically input text or images and its input and processing method, which can realize fully automatic identification, input and processing of images and text information such as business cards on the mobile phone.

The purpose of the present invention is achieved through the following technical solutions: a mobile phone that can automatically input text and images, including a camera, the camera signal output end of the camera is connected to the central processing unit in the mobile phone; the mobile phone is provided with a system master control module , camera driver module, text, image database module and display driver module, it is characterized in that:

The mobile phone is also provided with a layout processing and enhancement processing module, an intelligent layout analysis module, a character recognition module, a post-processing module, a data exchange module and an associated application module;

Under the control of the general control module of the system, the layout processing and enhancement processing module, the intelligent layout analysis module, the character recognition module and the post-processing module work sequentially to preprocess and enhance the camera signal, and classify according to the attributes of text and images Stored in the text and image database module, and then the data exchange module realizes the data exchange with the computer or with other mobile phones through the local network connection or remote network connection, and/or the display driver module displays the operation results on the mobile phone operation interface On the display screen of the mobile phone, and/or the associated application module performs associated application on the selected text and image information.

The purpose of the present invention is also achieved through the following technical solutions: a method for automatically entering and processing text and images in a mobile phone, characterized in that:

A. The mobile phone camera takes pictures of the text and images on the layout through the camera driver module, and obtains the electronic information of the text and images on the layout;

B. Carry out text and image preprocessing and enhancement processing on the text and image electronic information on the layout by the layout processing and enhancement processing module;

C. Analyze the layout structure by the intelligent layout analysis module, segment and extract various text areas and graphics areas, send the preprocessing and enhancement processing results of various text areas to the character recognition module, and preprocess and enhance the graphics area The result is sent to the graphics compression module, and the graphics compression module stores the compressed graphics data in the text and image database modules;

D. Perform character recognition on the preprocessing and enhancement processing results in various text areas by the character recognition module, and obtain character recognition results;

E. The character recognition result is corrected by the post-processing module, and the character recognition result is logically classified in combination with the logic structure pattern of the layout in the logic structure pattern library, and the classification result is stored in the text and image database modules;

F. Associate the classification results in the text and image database modules with the relevant applications of the mobile phone by the associated application module, and when starting a mobile phone application, call the text classification information associated with the application from the text and image database modules at the same time. The application; and/or according to the operation result of the mobile phone, the information in the text and image database module is displayed on the display screen of the mobile phone by the display driver module; The information in the described text and image database modules is exchanged with the computer or with other mobile phones.

The words and images on the layout in the step A are the contact information and the logo graphics on the business card.

The present invention uses the camera on the mobile phone to take pictures of the layout such as business cards, image them, and then perform image processing, intelligent layout analysis and character recognition, and then store the obtained business card information in a certain format in the database of the mobile phone. Application association, when a certain application is started, the business card information associated with the application is automatically called, and the corresponding application can be automatically executed according to the called information, such as telephone dialing, email, browser, etc., select A contact's business card can quickly realize the automatic dialing of the phone, send emails or browse the information of the other party's homepage, etc. Realize automatic input, identification and management of business card information on the mobile phone.

Adopt the mobile phone that the inventive method implements, compare with existing mobile phone with camera, have following advantage:

The business card information input can be realized by taking a single photo, without the need to input word by word, the speed is fast, the amount of input information is large, and the operation is convenient;

Graphical information such as the logo on the business card can be input at the same time, and stored in standard formats such as VCARD and XML, which is convenient for signal conversion between traditional media business cards and standard electronic business cards;

There is no need to store the text and images on the entire business card, and the necessary information on the business card can be extracted as needed and stored in standard electronic business card formats such as VCARD, which has low requirements for mobile phone storage space;

Without the use of a personal computer and a scanner, the full-automatic identification, entry and processing of business card information can be realized directly through the built-in camera on the mobile phone, without increasing the hardware cost of the mobile phone and without affecting its portability. The function of mobile phone business card recognition and entry has been added.

Description of drawings

Fig. 1 is the schematic diagram of the device outline structure of mobile phone of the present invention;

Fig. 2 is a schematic diagram of utilizing the mobile phone of the present invention to input and identify business card information;

Fig. 3 is a flow chart of fully automatic processing, analysis and identification of business card information;

Fig. 4 is a schematic diagram of the structural relationship of software modules for realizing the method of the present invention.

Detailed ways

The novel mobile phone proposed by the present invention utilizes a camera arranged on the mobile phone to realize full-automatic recognition, entry and management of text and image information such as business cards by the mobile phone.

The technology of the present invention will be further described by taking the processing of business card information as an example below.

Referring to Figure 1 and Figure 2, the built-in camera 1 on the mobile phone takes pictures of the business card 3 and displays it on the display screen 2. After image processing, intelligent layout analysis and character recognition, the business card information is stored in the database of the mobile phone in a certain format In the process, the automatic identification and entry of business card information is completed.

Referring to FIG. 3 , the figure shows the fully automatic identification and entry process of business card information.

Step 301, the camera picks up the image and text information on the business card, and the camera driver module takes pictures and images of the business card under the control of the central processing unit, so as to obtain the electronic information of the image and text on the business card;

In step 302, the image and text electronic information on the acquired business card are preprocessed by the layout processing and enhancement module. Since the business card is photographed with the built-in camera of the mobile phone, the background is complex, there are many irregular deformations, and the image quality is uneven, so it is necessary to first Perform layout preprocessing and enhancement processing, including four processes 302-a, 302-b, 302-c, and 302-d, wherein step 302-a is to perform Gaussian smoothing and denoising processing; step 302-b is to perform brightness and Contrast normalization processing to meet the information intake of business cards under different lighting conditions; 302-c is to perform background removal processing; 302-d is to perform projective transformation correction processing to meet the needs of business cards under different shooting angles (non-vertical angles) Input, the processing sequence of the above steps a, b, c and d can be carried out in the order shown in the figure, or not in this order, you can do all of them, or you can choose 1 to 3 of them to do;

Step 303: The layout analysis module intelligently analyzes the layout of the business card. After the mobile phone receives the image and text information of the business card, it performs intelligent layout analysis on the business card information, that is, separates and extracts different areas, including the name area, text area, and logo area. etc., determine the corresponding typesetting direction, layout size, position area on the business card, etc. of each area, and then send them into different modules according to the different attributes of these areas, such as sending the name area and text area into the character recognition (OCR) module, Send the logo area to the logo graphics compression module;

Step 304, character information identification is carried out to the name area and text area sent in by the character recognition (OCR) module, and then the result data after the identification is sent to the post-processing module;

Step 305, the image information in the logo area is compressed by the logo graphics compression module, and the compressed result is directly sent to the business card information database for storage;

The above steps 304 and 305 can be performed in parallel.

Step 306, the post-processing module corrects the result data after character recognition, and the correction is carried out in combination with the business card logical structure pattern library, including correction, classification processing, and the update process of the pattern library, specifically: the correction of the recognition result, for Confirm the recognition result of the character recognition (OCR) module, correct the error, and edit the result data; combine the logic structure pattern library of the business card to classify the business card information, and further identify the different areas after the intelligent layout analysis according to the character (OCR) module recognition results and business card logical structure pattern library, carry out logical classification, such as: name, address, zip code, telephone, job title, email, website, etc.; if you encounter a business card logical structure that cannot be processed by the business card logical structure pattern library Pattern, the logical structure of the business card can be added to the pattern library to update the pattern library;

In step 307, the database module sends the business card classification information and logo graphic compression data obtained by the post-processing module into the business card information database in the prescribed format such as VCARD and XML for retrieval and output.

The system maintains two databases: contact card information database and business card logical structure pattern library.

Steps 308, 309, 310, call the business card information database according to user needs, and display the business card information on the mobile phone display screen by the display driver module (the display driver module on the original mobile phone can be used) through the mobile phone operation interface; , through local connection or remote network connection, exchange business card data with computer, or exchange business card data with other mobile phones; and use the associated application module to link business card information with other related applications such as: dial-up program, email sending program, browser program For example, by selecting a contact's business card, you can automatically and quickly dial a number, send an email, or browse the information on the other party's homepage.

When the present invention is implemented, the camera driver module (mobile phone with camera) and the display driver module of the original mobile phone can be utilized, and technologies such as image, word processing, recognition and layout analysis that have become mature at present can be combined into the mobile phone , so as to realize the automatic entry, classification and related applications of business card information with mobile phones.

Referring to FIG. 4 , it shows the structural relationship of the software modules involved in realizing the object of the present invention. Including camera driver module 401, layout processing and enhancement module 402, layout analysis module 403, character recognition module 404, text post-processing module 405, database module 406, business card data exchange module 407, display driver module 408, associated application module 409 and system General control module 410 . The system master control module 410 completes the scheduling of each module and the switching between modules. The system master control module 410 can use the original system module of the mobile phone, but the function of the original system module needs to be enhanced.

The layout analysis module sends the preprocessing and enhancement processing results in the segmented text area to the character recognition module 404, and compresses the segmented Logo graphics data and directly sends them to the database module for storage. In the database module 406, there is a text information base of business cards and a logical structure pattern base of business cards, and the post-processing module utilizes the existing patterns in the logical structure pattern base to guide the classification of text information on business cards, and sends the text classification results to the text information base of business cards for preservation , when the existing patterns in the logical structure pattern library cannot guide the classification of the text information on the current business card, store the logical structure pattern of the current business card in the logical structure pattern library, and update the logical structure pattern library of the business card.

The business card data exchange module 407, the display driver module 408, and the associated application module 409 call corresponding text classification information and Logo graphics information from the database module according to user needs.

The new mobile phone provided by the invention realizes automatic identification, input and management of business card information on the mobile phone directly through the built-in camera on the mobile phone, and can quickly and conveniently input the business card information of contacts into the mobile phone, thereby facilitating users Communication with contacts.

For the convenience of description, take the mobile phone to obtain business card information as an example, but it is not limited to business cards. Any equivalent modification or change made to the present invention without departing from the principle of the present invention shall be included in the protection scope of the present invention.

Claims (10)

1. mobile phone of typing literal, image automatically, comprise camera, the image pickup signal output of camera is connected with the central processing unit in the mobile phone, described mobile phone is provided with system's total control module, webcam driver module, literal, view data library module and driver module, it is characterized in that:

Described mobile phone also is provided with the space of a whole page and handles and enhancement process module, intelligent printed page analysis module, character recognition module, post-processing module, data exchange module and associated application module;

Under the control of system's total control module, handle and the enhancement process module by the space of a whole page, intelligence printed page analysis module, character recognition module and post-processing module sequential working, image pickup signal is carried out preliminary treatment and enhancement process, with press literal, the attributes of images classification is stored in literal, in the view data library module, connect by local network by data exchange module again or telecommunication network connect realize and computer between or and other mobile phone between exchanges data, and/or by the mobile phone operation interface operating result is presented on the mobile phone display screen, and/or by the literal of associated application module to choosing by driver module, image information is carried out associated application.

2. automatic typing of mobile phone and the method for handling literal, image is characterized in that:

A. mobile phone cam with the literal on the space of a whole page, image photographic imaging, obtains the electronic information of literal on the space of a whole page, image by the webcam driver module;

B. handle with the enhancement process module by the space of a whole page literal on the space of a whole page, image electronic information are carried out literal, image preliminary treatment and enhancement process;

C. by intelligent printed page analysis module layout structure is analyzed, cut apart and extract various character areas and graphics field, send character recognition module with the preliminary treatment and the enhancement process result of various character areas, send the figure compression module with the preliminary treatment and the enhancement process result of graphics field, the graph data after the figure compression module will compress deposits in literal, the view data library module;

D. by character recognition module character recognition is carried out in preliminary treatment in the various character areas and enhancement process result, obtained character identification result;

E. by post-processing module character identification result is revised, and character identification result is carried out logical division, and classification results is deposited in literal, the view data library module in conjunction with the logic structural pattern of the space of a whole page in the logic structural pattern storehouse;

F. by the associated application module classification results in literal, the view data library module and the related application of mobile phone are associated, when starting a kind of mobile phone and using, from literal, view data library module, call this application of script classify information and executing that is associated with this application simultaneously; And/or, the information in literal, the view data library module is presented on the mobile phone display screen by driver module according to the mobile phone operation result; And/or by data exchange module, connect or the telecommunication network connection by local network, information in described literal, the view data library module and computer are carried out exchanges data or carried out exchanges data with other mobile phone.

3. automatic typing of mobile phone according to claim 2 and the method for handling literal, image, it is characterized in that: the literal in the described steps A on the space of a whole page, image are associated person information and the Logo figures on the business card.

4. automatic typing of mobile phone according to claim 2 and the method for handling literal, image, it is characterized in that: preliminary treatment and enhancement process among the described step B further comprise:

B1. Gauss's smoothing denoising sonication;

B2. the normalized of brightness and contrast;

B3. background removal is handled;

B4. projective transformation treatment for correcting.

5. automatic typing of mobile phone according to claim 4 and the method for handling literal, image is characterized in that: the order of pressing B1, B2, B3, B4 is carried out Overall Steps; Perhaps, carry out Overall Steps not according to the order of B1, B2, B3, B4; Perhaps, from B1, B2, B3, B4, select execution one to three step wherein.

6. automatic typing of mobile phone according to claim 2 and the method for handling literal, image is characterized in that:

Among the described step C, cut apart and extract each character area and graphics field, also comprise and determine each regional writing direction, space of a whole page size and each regional position in the space of a whole page that each is regional.

7. automatic typing of mobile phone according to claim 2 and the method for handling literal, image is characterized in that:

In the described step e, described correction is that character identification result is confirmed and the mistake of editing and revise discovery; Described logical division is each character area that step C is partitioned into, according to the character identification result of step D acquisition and the logic structural pattern storehouse of the space of a whole page, logical division is the classified information that comprises name of contact person, address, postcode, telephone number, post, mailbox, network address.

8. automatic typing of mobile phone according to claim 2 and the method for handling literal, image is characterized in that:

In the described step e, comprising that also the renewal in the logic structural pattern storehouse of the space of a whole page is handled, is in the time of can not utilizing existing layout mode to instruct logical division in the logic structural pattern storehouse of the space of a whole page, current layout mode to be deposited in the logic structural pattern storehouse of the space of a whole page.

9. automatic typing of mobile phone according to claim 2 and the method for handling literal, image, it is characterized in that: in the described step F, described association comprises the Phone Dialer of mobile phone, send Email program, browse the program of the other side's home tip is associated with classification results in literal, the view data library module, choose the contact person, promptly realize dialing function or send Email or browse the other side's homepage function.

10. automatic typing of mobile phone according to claim 2 and the method for handling literal, image, it is characterized in that: among the described step C, deposit the graph data in literal, the view data library module in, with the classification results that deposits in the described step e in literal, the view data library module, be to store by the prescribed form that comprises VCARD form, XML form.

Images