CN110717397A - An online translation system based on mobile phone camera - Google Patents

Abstract

The invention discloses an online translation system based on a mobile phone camera, which can automatically focus to obtain pictures; image preprocessing; after extracting corresponding text information, use a cube text recognition engine provided by Google to recognize text; The translated text, pass the text to be translated to the translation module to return the translation result. The beneficial effect of the present invention is that accurate results are obtained by successful identification and translation.

Description

An online translation system based on mobile phone camera

technical field

The invention belongs to the technical field of online translation, and relates to an online translation system based on a mobile phone camera.

Background technique

With the development of the economy, more and more people choose to travel abroad, but the language barrier has always plagued people. Therefore, a software that can recognize and translate text in tourist scenes will be able to meet the needs of domestic and foreign tourists. The travel translation assistant has OCR real-time recognition and translation functions, which can help tourists solve the problem of language barriers during travel. The travel translation assistant realizes the recognition and translation function of English and Chinese in different scenarios based on the Android platform. The application is developed based on the Java language, installed on an Android phone, and can be debugged in a variety of natural scenarios. Test scenarios include book document recognition and translation, menu recognition and translation, street sign identification and translation, and so on. Common translation software requires manual input of the text to be translated before translation. However, for the convenience of users, this travel translation assistant has designed the function of direct translation after word recognition by the camera, which saves the trouble of manually inputting the translation text.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide an online translation system based on a mobile phone camera, and the beneficial effect of the present invention is to successfully identify and translate to obtain accurate results.

The technical scheme adopted in the present invention is to carry out according to the following steps:

Step 1: Auto-focus to acquire pictures; the camera automatically focuses on the content to be recognized every two seconds and acquires pictures, so as to dynamically locate and acquire images through auto-focus;

Step 2: Image preprocessing; the steps of image preprocessing are: scan image --> obtain image --> analyze image --> obtain image parameters --> image denoising --> calculate image threshold --> pair Image grayscale binarization and save grayscale value --> get grayscale image;

Step 3: Recognition function, after extracting the corresponding text information, use the cube text recognition engine provided by Google to recognize the text;

Step 4: Translation function, after the word recognition is completed, the text to be translated is obtained, and the translation result can be returned by passing the text to be translated to the translation module.

Further, in step 1, first set a timer Timer in the system, it will execute a TimerTask at regular intervals, and execute the focus method in the TimerTask, the Camera class provides an autofocus method, which receives the AotoFocusCallback callback, After this method is executed, the camera will automatically focus. When the focus is completed, the callback method is triggered and the Timer is started again to achieve continuous automatic focus. Finally, the camera autofocus method is designed. After judging that the focus is successful, a message is sent to indirectly control the execution of the photo. Get pictures.

Further, step 2 includes

(1) Image acquisition: by calling the camera, scan to obtain a fixed rectangular area image, after obtaining the image, analyze the image, and obtain the parameters of the image to be processed;

(2) Image de-noising, the de-noising algorithm used by the system is to obtain the median value of all pixel values in the fixed size area, and take a table area of 3 times 3, pixel[0~8] is a one-dimensional pixel Value sequence, the center point is pixel[4], sort the values of 9 pixel points, then take the middle value, and then store it in pixel[4], that is, the center point, to obtain the pixel value after denoising;

(3) Calculate the threshold value. In the system, the one-dimensional maximum entropy method, the big law and the iterative method are used to calculate the image threshold value respectively, and the optimal threshold value result is obtained by comprehensive comparison;

(4) Grayscale binarization processing, the system performs grayscale binarization processing on the image according to the previously calculated optimal threshold;

(5) Extract features, segment the preprocessed image, extract image features regionally, and extract text information in the image. The text extraction method in the system uses the data transfer module between activities in the Android system. The image is placed in the Activity module where the photo is taken for processing. At the same time, the function of taking words is completed in this module, and then the result obtained from the image processing is returned to the Activity of the translation module, and then the text is recognized and translated.

Further, step 3 includes

(1) The cube text recognition engine is divided into two parts: image layout analysis and character segmentation and recognition;

①The image layout analysis mainly analyzes the connected area, which is the preparation work for character recognition. Through a hybrid page layout analysis method based on tab stop detection, the table, text, picture and other contents of the image are distinguished;

②The process of character segmentation and recognition is to first find the block area, locate the text line and word, identify the text, first perform the first character recognition, determine the character area type, and compare the character library according to the determination result. For text characters, the glued characters are divided, and the wrongly divided characters are merged to complete the fine segmentation of the characters;

(2) Cube text recognition engine training:

Use the jTessBoxEditor tool to train the language font library. The task to be completed in this system is to recognize and translate Chinese and English. Therefore, in the training process, the following data packets in tessdata are trained to obtain the language applied in the system. Recognition library:

chi_sim.traineddata,

chi_tra.traineddata, eng.traineddata, chi_tra_vert.traineddata, chi_sim_vert.traineddata, when using this system for the first time, you need to load the recognition library first, and load the recognition language library into the mobile phone storage root directory to run the recognition function correctly, which is obtained by calling the training language recognition library.

Further, step (4) includes

①Using floating-point algorithm to grayscale the scanned area image, the image scanned by the mobile phone camera is usually a 24-bit deep RGB image, and an 8-bit deep grayscale image is obtained after grayscale processing;

② After the image is processed by ordinary binarization, a black and white image is obtained. The system saves the grayscale information of the image when using the binarization process. According to the different pixel values of different images, the grayscale information is binarized, and the image It is classified as an image with only two grayscale values, and the final image is a grayscale image;

③ After the grayscale binarization process is performed, first determine whether the number of dark value pixels is more than the number of light value pixels, and if the number of dark value pixels is more than the number of light value pixels, perform inverse color processing.

Further, the translation function flow in step 4 is start-->acquire the text information obtained by recognition-->encode the text information-->MD5 encryption, generate a signature-->transmit information-->obtain the translation result-->in The translation result is displayed on the program interface --> End.

Detailed ways

The present invention will be described in detail below with reference to specific embodiments.

Example:

Step 1: Autofocus to get the picture;

Autofocus: The camera automatically focuses on the content to be recognized and acquires a picture every two seconds, thereby dynamically positioning and acquiring images through autofocus. First, a timer Timer is set in the system, which will execute a TimerTask at regular intervals, and execute the focus method in the TimerTask. The Camera class provides the autofocus method, which receives the AotoFocusCallback callback. After this method is executed, the camera will automatically focus, and when it focuses (success or failure), the callback method is triggered and the Timer is started again to achieve continuous interval autofocus. Finally, the camera auto-focus (raiseEvent_OnAutoFocusSuccess) method is designed to send a message after judging that the focus is successful, and indirectly control the execution of taking pictures to obtain pictures.

Step 2: Image preprocessing;

The steps of image preprocessing are: scan the image --> obtain the image --> analyze the image --> obtain the image parameters --> image denoising --> calculate the image threshold --> binarize the image gray level and Save Grayscale Values --> Get Grayscale Image.

(1) Image acquisition: By calling the camera, the fixed rectangular area image is obtained by scanning. After obtaining the image, parse the image to obtain the parameters of the image to be processed, such as color value, bitmap line width, and number of pixels. This is the first step in image processing.

(2) Image denoising. The denoising algorithm adopted by this system is to obtain the median value of all pixel values in the set fixed size area. Take a 3 by 3 table area. pixel[0～8] is a one-dimensional sequence of pixel values, and the center point is pixel[4]. Sort the values of the 9 pixel points, then take the middle value, and then store it in pixel[4], that is, the center point, to obtain the pixel value after denoising.

(3) Calculate the threshold. In this system, the one-dimensional maximum entropy method, the big law and the iterative method are used to calculate the image threshold respectively, and the optimal threshold result is obtained by comprehensive comparison.

(4) Grayscale binarization processing. The system performs grayscale binarization on the image according to the previously calculated optimal threshold.

①Using floating-point arithmetic to grayscale the scanned area image. The image scanned by the mobile phone camera is usually a 24-bit deep RGB image, and an 8-bit deep grayscale image is obtained after grayscale processing.

② After the image is processed by ordinary binarization, a black and white two-color image is obtained. It has only two colors, black and white, that is to say, the bit depth of each pixel is 1, which occupies one binary bit. However, the system preserves image grayscale information when using binarization. According to different pixel values of different images, the grayscale information is binarized, and the picture is classified as an image with only two grayscale values. The final image is a grayscale image, but after grayscale binarization, the image recognition accuracy is higher.

③ After the grayscale binarization process, it can be judged whether the number of dark value pixels is more than the number of light value pixels, and if the number of dark value pixels is more than the number of light value pixels, the inverse color processing is performed.

(5) Extract features. The preprocessed image is segmented, the image features are extracted by regionalization, and the text information in the image is extracted. The method of text extraction in the system uses the data transfer module between activities in the Android system. First, the image is placed in the Activity module where the photo is taken for processing, and the function of word extraction is completed in this module, and then the result of the image processing is returned to the Activity of the translation module, and then the text is recognized and translated.

Step 3: Identify the function;

After extracting the corresponding text information, the system uses the cube text recognition engine (Tesseract-ocr) provided by Google to recognize the text.

(1) Principle of Tesseract-ocr: The Tesseract engine can be divided into two parts: image layout analysis and character segmentation and recognition.

①Picture layout analysis mainly analyzes the connected area, which is the preparation for character recognition. It uses a hybrid tab-stop detection-based page layout analysis method to differentiate images from tables, texts, pictures, etc.

②The process of character segmentation and recognition is to first find the block area, locate the text lines and words, and recognize the text. The first character recognition is carried out, and the characters are recognized by comparing the character library according to the judgment result by judging the type of the character area. Then, according to the recognized text characters, the sticky characters are segmented, and the wrongly segmented characters are merged at the same time to complete the fine segmentation of the characters.

(2) Cube text recognition engine (Tesseract-ocr) training:

Use the jTessBoxEditor tool to train language fonts. The task we need to complete in this system is the recognition and translation of Chinese and English, so during the training process, the following data packets in tessdata are trained to obtain the language recognition library used in the system:

chi_sim.traineddata,

chi_tra.traineddata, eng.traineddata, chi_tra_vert.traineddata, chi_sim_vert.traineddata. When using this system for the first time, you need to load the recognition library first, and load the recognition language library into the root directory of the mobile phone storage to run the recognition function correctly. By calling the language recognition library obtained by training, text information can be recognized accurately and efficiently.

Step 4: Translate function;

After the word recognition is completed, the text that needs to be translated is obtained. We pass the text to be translated to the translation module to return the translation result. The translation function process is start --> get the recognized text information --> encode and convert the text information -->MD5 encryption, generate signature-->Transmission information-->Get translation result-->Display translation result on the program interface-->End.

In this system, in order to ensure the security of the calling interface, the interface adopts IP restrictions and encrypts the transmission data through MD5 encoding, and interacts with the server. First, the information to be translated is encrypted with MD5, a signature is generated for verification, and the information to be translated is transmitted to the server through HTTP Post. After the translation result is returned by the server in Json format, the returned XML information is parsed, and the final translation result is presented in the translation result column in a common representation form, thus realizing the translation function. Message Digest Algorithm 5 (MD5) is a hash function widely used in the field of computer security to provide integrity protection of messages. It is used to ensure complete and consistent information transmission and is one of the hash algorithms widely used by computers.

The system of the invention not only has the function of real-time word selection and recognition, but also provides the function of online translation, which makes the user experience better and provides convenience for the user in daily life. The main technical fields of the system are pattern recognition and natural language processing. It mainly realizes real-time text information recognition and translation functions, and is a good translation assistant for people when traveling. The system completes the design of each sub-module through image processing technology, feature extraction technology, text recognition technology, translation technology and other technical means, and realizes the function of direct translation after word recognition by the camera, eliminating the need for people to manually input translation texts. trouble. The system has broad application prospects and important practical significance.

The above is only a preferred embodiment of the present invention, and does not limit the present invention in any form. Any simple modifications, equivalent changes and modifications made to the above embodiments according to the technical essence of the present invention belong to the present invention. within the scope of the technical solution of the invention.

Claims (6)

1. an online translation system based on mobile phone camera is characterized in that carrying out according to the following steps: Step 1: Auto-focus to acquire pictures; the camera automatically focuses on the content to be recognized every two seconds and acquires pictures, so as to dynamically locate and acquire images through auto-focus; Step 2: Image preprocessing; the steps of image preprocessing are: scan image --> obtain image --> analyze image --> obtain image parameters --> image denoising --> calculate image threshold --> pair Image grayscale binarization and save grayscale value --> get grayscale image; Step 3: Recognition function, after extracting the corresponding text information, use the cube text recognition engine provided by Google to recognize the text; Step 4: Translation function, after the word recognition is completed, the text to be translated is obtained, and the translation result can be returned by passing the text to be translated to the translation module. 2. according to a kind of online translation system based on mobile phone camera according to claim 1, it is characterized in that: In the step 1, a timer Timer is first set in the system, it will execute a TimerTask at regular intervals, and execute the focus method in the TimerTask, the Camera class provides an automatic focus method, which receives the AotoFocusCallback callback, After this method is executed, the camera will automatically focus. When the focus is completed, the callback method is triggered and the Timer is started again to achieve continuous automatic focus. Finally, the camera autofocus method is designed. After judging that the focus is successful, a message is sent to indirectly control the execution of the photo. Get pictures. 3. a kind of online translation system based on mobile phone camera according to claim 1, is characterized in that: described step 2 comprises: (1) Image acquisition: by calling the camera, scan to obtain a fixed rectangular area image, after obtaining the image, analyze the image, and obtain the parameters of the image to be processed; (2) Image de-noising, the de-noising algorithm used by the system is to obtain the median value of all pixel values in the fixed size area, and take a table area of 3 times 3, pixel[0~8] is a one-dimensional pixel Value sequence, the center point is pixel[4], sort the values of 9 pixel points, then take the middle value, and then store it in pixel[4], that is, the center point, to obtain the pixel value after denoising; (3) Calculate the threshold value. In the system, the one-dimensional maximum entropy method, the big law and the iterative method are used to calculate the image threshold value respectively, and the optimal threshold value result is obtained by comprehensive comparison; (4) Grayscale binarization processing, the system performs grayscale binarization processing on the image according to the previously calculated optimal threshold; (5) Extract features, segment the preprocessed image, extract image features regionally, and extract text information in the image. The text extraction method in the system uses the data transfer module between activities in the Android system. The image is placed in the Activity module where the photo is taken for processing. At the same time, the function of taking words is completed in this module, and then the result obtained from the image processing is returned to the Activity of the translation module, and then the text is recognized and translated. 4. according to a kind of online translation system based on mobile phone camera according to claim 3, it is characterized in that: described step (4) comprises ①Using floating-point algorithm to grayscale the scanned area image, the image scanned by the mobile phone camera is usually a 24-bit deep RGB image, and an 8-bit deep grayscale image is obtained after grayscale processing; ② After the image is processed by ordinary binarization, a black and white image is obtained. The system saves the grayscale information of the image when using the binarization process. According to the different pixel values of different images, the grayscale information is binarized, and the image It is classified as an image with only two grayscale values, and the final image is a grayscale image; ③ After the grayscale binarization process is performed, first determine whether the number of dark value pixels is more than the number of light value pixels, and if the number of dark value pixels is more than the number of light value pixels, perform inverse color processing. 5. a kind of online translation system based on mobile phone camera according to claim 1, is characterized in that: described step 3 comprises: (1) The cube text recognition engine is divided into two parts: image layout analysis and character segmentation and recognition; ①The image layout analysis mainly analyzes the connected area, which is the preparation work for character recognition. Through a hybrid page layout analysis method based on tab stop detection, the table, text, picture and other contents of the image are distinguished; ②The process of character segmentation and recognition is to first find the block area, locate the text line and word, identify the text, first perform the first character recognition, determine the character area type, and compare the character library according to the determination result. For text characters, the glued characters are divided, and the wrongly divided characters are merged to complete the fine segmentation of the characters; (2) Cube text recognition engine training: Use the jTessBoxEditor tool to train the language font library. The task to be completed in this system is to recognize and translate Chinese and English. Therefore, in the training process, the following data packets in tessdata are trained to obtain the language applied in the system. Recognition library: chi_sim.traineddata, chi_tra.traineddata, eng.traineddata, chi_tra_vert.traineddata, chi_sim_vert.traineddata, when using this system for the first time, you need to load the recognition library first, and load the recognition language library into the mobile phone storage root directory to run the recognition function correctly, which is obtained by calling the training language recognition library. 6. according to the described a kind of online translation system based on mobile phone camera according to claim 1, it is characterized in that: in described step 4, the translation function process flow is to start --> obtain the text information obtained by recognition --> code conversion of text information- ->MD5 encryption, generate signature-->Transmission information-->Get translation result-->Display translation result on the program interface-->End.