CN106547891A - For the quick visualization method of the pictured text message of palm display device - Google Patents

Abstract

The present invention provides a fast visualization method for pictorial text information aimed at a palm display device, comprising the following steps: (1) obtaining a picture with text information through a palm display device; (2) converting the picture in step (1) to (3) The user selects the desired character segment from the isolated character segment obtained in step (2), and combines the selected character segment with the desired dimension Correspond; (4) Visualize the dimensions and corresponding data obtained in step (3); the method of the present invention makes up for the gap in the research field of rapid visualization of pictorial text information, allowing more people to use visualization The tool is used to speed up the understanding of data and to better disseminate data. At the same time, the present invention also enriches the channels for us to obtain data, making more data in the real world readily available.

Description

A fast visualization method for pictorial text information on handheld display devices

technical field

The invention relates to the technical field of data visualization, in particular to a fast visualization method for pictorial text information of a handheld display device.

Background technique

In our daily life, we come across a lot of pictorial text information, such as nutritional information on milk packaging, articles in books, and phone bill package information on posters in mobile business halls. It is usually inefficient to directly obtain the effective information in the text by reading the text, especially when multiple data are compared with each other.

Visualization is one of the most effective tools to solve such problems. Visualization is the theory, method and method of using computer graphics and image processing technology to convert data into graphics or images and display them on the screen for interactive processing. technology. It involves many fields such as computer graphics, image processing, computer vision, and computer-aided design, and has become a comprehensive technology for studying a series of issues such as data representation, data processing, and decision analysis. The virtual reality technology that is currently developing rapidly is also based on the visualization technology of graphics and images.

Visualization can quickly help people understand the effective information contained in the data, and it can also effectively assist people in comparing multiple sets of data. However, most of the existing visualization tools only support digital data, and cannot directly support the visualization of the above-mentioned pictorial information in the real world. To visualize the above-mentioned pictorial information, it is necessary to first digitize the characters with the help of character recognition tools, then convert them into structured data that can be used by visualization tools, and finally use visualization tools to visualize the structured data. Completing the above steps requires professional knowledge in the field of computer and visualization, which greatly increases the threshold for ordinary users to use visualization technology.

OCR (Optical Character Recognition, Optical Character Recognition) means that an electronic device (such as a scanner or a digital camera) checks characters printed on paper, determines its shape by detecting dark and bright patterns, and then uses character recognition to translate the shape into a computer The process of text; that is, for printed characters, the text in the paper document is converted into a black and white dot matrix image file by optical means, and the text in the image is converted into a text format by recognition software for further processing by word processing software Editing technology. How to debug or use auxiliary information to improve the accuracy of recognition is the most important topic of OCR, and the term ICR (Intelligent Character Recognition) was born because of this. The main indicators to measure the performance of an OCR system are: rejection rate, false recognition rate, recognition speed, user interface friendliness, product stability, usability and feasibility, etc.

On the other hand, with the rapid development of the smart phone field, the high-resolution camera and display screen, strong CPU and sensitive touch response of the current smart phone have enabled it to support many things that were previously only available on personal computers. task to be completed. We can use the camera function of smartphones to obtain various information in the real world, and then process the information with the help of its powerful processor, and then display and share the information.

To sum up, there is currently no method and device for effectively visualizing data on entity images.

Contents of the invention

The present invention provides a fast visualization method for pictorial text information aimed at a handheld display device, which makes up for the gap in the rapid visualization of pictorial text information, thereby more effectively disseminating data and enriching channels for obtaining data.

A fast visualization method for pictorial text information of a handheld display device, characterized in that it comprises the following steps:

(1) Obtain a picture with text information through a handheld display device;

Use a handheld display device, such as a smart phone, to take pictures of the pictured text information, obtain the original picture with the text information, and copy the data from the physical world to the mobile phone storage.

(2) converting the pictorial text information in step (1) into digitized text information to obtain isolated character fragments;

(3) The user selects desired character segments from the isolated character segments obtained in step (2), and corresponds the selected character segments to desired dimensions;

(4) Graphically visualize the dimension and corresponding data obtained in step (3).

In order to be applicable to common life fields, preferably, in step (3), the dimension includes name, type and corresponding unit.

In order to facilitate the user's operation, preferably, in step (3), the desired dimension is determined by the user's choice, automatically judged by the judgment module prestored in the palm display device, or automatically judged by the judgment module prestored in the palm display device and then determined by the user. .

In order to facilitate the implementation of the method of the present invention, preferably, in step (2), the imaged text information is converted into digitized text information by optical character recognition software.

In order to improve the extraction effect of the picture text, preferably, in step (2), before converting the pictured text information in step (1) into digitized text information, the picture obtained in step (1) is corrected.

Further preferably, the image obtained in step (1) is corrected using an open source computer vision library.

Specifically, step (2) can be completed in the following ways:

The user uploads the picture obtained in step (1) to the cloud server, and then, the server uses OpenCV (Open Source Computer Vision Library, which is a cross-platform computer vision library) to correct the picture (correct the position of the characters, improve clarity of characters, etc.) to improve the accuracy of OCR recognition; then, the server uses Optical Character Recognition (OCR, Optical Character Recognition) technology to convert the characters in the picture into digital character information that can be recognized by the computer, and the recognized The result is sent back to the user; finally, after seeing the recognition result, the user can further correct the wrong recognition result manually on the basis of OCR recognition;

Specifically, step (3) can be completed in the following ways:

Step (2) will generate some isolated character fragments. The user can first select a dimension (two or three dimensions are generally selected to correspond to the xy coordinate system and the xyz coordinate system), and then arbitrarily select some of the fragments as the corresponding dimension data, specify the basic information such as the name, type, and unit of their corresponding dimensions.

The method of the present invention can also design some auxiliary strategies to speed up the user's selection of data. At the same time, for a fragment set selected by the user, it will automatically pre-judge its type and unit according to the data content contained in the fragment in the collection. If some commonly used units are recognized, it also supports conversion between units, thereby speeding up User settings for a single dimension. In addition, our method allows users to perform algebraic operations on existing data dimensions to generate new dimensions.

Beneficial effects of the present invention:

It fills the gap in the research field of fast visualization of pictorial text information, and builds a bridge between text information in the physical world and visualization technology in the digital world. After promotion, it can greatly increase the audience of visualization technology and let more people People can use visualization tools to speed up the understanding of data and to better disseminate data. At the same time, the present invention also enriches the channels for us to obtain data, making more data in the real world readily available.

Description of drawings

FIG. 1 is a flow chart of the method for quickly visualizing pictorial text information on a handheld display device according to the present invention.

Figure 2 is the original picture with text information obtained through a smart phone.

Figure 3 is some isolated character fragments that the user gets.

FIG. 4 is a schematic diagram of the result of the user selecting the character segment in FIG. 3 .

Fig. 5 is a schematic diagram of the property setting interface of the dimension with colored background selected in Fig. 4 .

FIG. 6 is a schematic diagram of the attribute setting interface of the dimension with colored background selected in FIG. 4 .

FIG. 7 is a schematic diagram of the results of selecting rows for the dimensions of the colored background selected in FIG. 4 .

FIG. 8 is a schematic diagram of the unit conversion result of the dimensions of the colored background selected in FIG. 7 .

Fig. 9 is a schematic diagram of the initial interface for performing algebraic operations on the two dimensions selected in Fig. 8 .

FIG. 10 is a schematic diagram of an algebraic operation performed on the two dimensions selected in FIG. 8 .

Fig. 11 is a schematic diagram of the result of the algebraic operation in Fig. 10 .

FIG. 12 is a schematic diagram of a gesture for entering the selection mode of the horizontal axis dimension.

Fig. 13 is a schematic diagram of a gesture for entering the vertical axis dimension selection mode.

Fig. 14 is a schematic diagram of a gesture of a reset operation.

Fig. 15 is a schematic diagram of a gesture for adding a group operation.

FIG. 16 is a schematic diagram of the histogram visualization result of the data in FIG. 11 .

FIG. 17 is a schematic diagram of the line chart visualization result of the data in FIG. 11 .

FIG. 18 is a schematic diagram of the result of modifying the title of the visualization result in FIG. 17 .

detailed description

The present invention will be described in detail below in conjunction with the accompanying drawings.

The fast visualization method for the pictorial text information of the handheld display device in this embodiment,

As shown in Figure 1, it includes the following steps:

Step 1: Use a smartphone to obtain textual information in pictures:

Take pictures of the pictorial text information through the smartphone to obtain the original picture with the text information, as shown in Figure 2, and copy the data from the physical world to the mobile phone storage;

Step 2: Use OCR technology to recognize the characters in the obtained picture, the specific process is as follows:

First, the user uploads the pictures obtained by taking photos to the cloud server; then, the server uses OpenCV (Open Source Computer Vision Library, which is a cross-platform computer vision library) to correct the pictures (correct the position of the characters, improve the accuracy of the characters). clarity, etc.) to improve the accuracy of OCR recognition; then, the server uses Optical Character Recognition (OCR, Optical Character Recognition) technology to convert the characters in the picture into digital character information that can be recognized by the computer, and returns the recognition result to Pass it to the user; finally, after seeing the recognition result, the user can manually further correct the wrong recognition result on the basis of OCR recognition;

Step 3: Structure the results of OCR recognition:

Step 2 can recognize the characters in the picture, but cannot automatically obtain the association between these character fragments, and what the user gets is only some isolated character fragments, as shown in Figure 3.

First, the user first selects the dimension, as shown in Figure 3. The first two icons at the bottom correspond to the two dimensions of the vertical axis and the horizontal axis respectively;

Then arbitrarily select some fragments as the data of the selected dimension, and specify the basic information such as the name, type, and unit of their corresponding dimension. As shown in 4 and 6, those character fragments with colored background (the light-colored part in the figure, which can actually be set to any color, such as yellow) are selected, and we specify the attribute name as "expense" and the attribute category as " Category type", the unit is "$".

This method also has an auxiliary strategy to speed up the user's selection of data. The steps of interlaced selection are as follows:

After selecting the colored background in Figure 4 (the dark part in the figure, it can actually be set to any color to distinguish the cost column, such as blue), set the starting point to 0 (starting from 0), and every 2 Select one of the segments (as shown in Figure 5), and the effect of interlaced selection as shown in Figure 7 is obtained.

At the same time, for a fragment set selected by the user, it can also automatically pre-judge its type and unit according to the data content contained in the fragment in the collection. If some commonly used units are recognized, it also supports conversion between units, thereby speeding up The user sets a single dimension, and the automatic judgment steps are as follows:

As shown in Figure 5, the built-in recognition system automatically judges that the selected data is numerical, extracts the units m and g and gives the conversion ratio between the two. In addition, this method also allows users to perform algebraic operations on existing data dimensions to generate new dimensions. As shown in Figures 9-11, by dividing the cost dimension and the traffic dimension, we can obtain the unit price dimension of network traffic.

In addition, in view of the characteristics of small icons on the mobile phone screen and easy misclicking, the method of the present invention also adds support for gesture operations. As shown in Figures 12 to 15, the dot indicates the starting point of the gesture, and the four gestures respectively enter the horizontal axis dimension selection mode , enter the vertical axis dimension selection mode, reset, and add grouping four operations.

Step 4: Design the final visualization

According to the data types bound to the two dimensions in step 3, the visualization scheme and visual coding options are automatically recommended for users to select, modify and share. As shown in Figures 16-18, this method recommends two visual encoding methods for the data, bar chart and line chart, and provides a variety of color schemes for users to choose, and users can also modify the title of this visualization work.

By combining the visualization tasks of smartphones and pictorial text information, the present invention makes up for the gap in the research field of rapid visualization of pictorial text information, and bridges the gap between text information in the physical world and the digital world. A bridge between visualization technologies. Through the application of the present invention, users only need a smart phone to complete all the steps from the pictorial text information to the final visualization work, which greatly reduces the threshold of visualization applications, and can greatly increase the audience of visualization technology after promotion, allowing more More people can use visualization tools to speed up data understanding and better disseminate data. At the same time, the present invention also enriches the channels for us to obtain data, making more data in the real world readily available.

Claims (6)

1. a kind of quick visualization method of the pictured text message for palm display device, it is characterised in that include with Lower step：

(1) picture with text message is obtained by palm display device；

(2) the pictured text message in step (1) is changed into into digitized text message, obtains isolated character sheet Section；

(3) desired character fragments are chosen in the isolated character fragments that user is obtained from step (2), and the word that selection is obtained FUPIAN section is mapped with desired dimension；

(4) dimension and corresponding data that obtain in step (3) are carried out into graph visualization.

2. the quick visualization method of the pictured text message of palm display device is directed to as claimed in claim 1, and which is special Levy and be, in step (3), described dimension includes title, type and corresponding unit.

3. the quick visualization method of the pictured text message of palm display device is directed to as claimed in claim 1, and which is special Levy and be, in step (3), desired dimension is that user selects the judge module for determining, prestoring by palm display device automatic Judge or select to determine by user after the judge module automatic decision that palm display device prestores again.

4. the quick visualization method of the pictured text message of palm display device is directed to as claimed in claim 1, and which is special Levy and be, in step (2), digitized text envelope is changed into pictured text message by optical character recognition Breath.

5. the quick visualization method of the pictured text message of palm display device is directed to as claimed in claim 1, and which is special Levy and be, in step (2), the pictured text message in step (1) is changed into before digitized text message, to step (1) obtain picture to be corrected.

6. the quick visualization method of the pictured text message of palm display device is directed to as claimed in claim 5, and which is special Levy and be, picture is obtained to step (1) and is corrected using computer vision storehouse of increasing income.