CN115294571A - Character recognition method of character scanning image, intelligent watch and electronic equipment - Google Patents

Abstract

The invention discloses a text recognition method for text scanning images, a smart watch and an electronic device. The text recognition method for text scanning images is characterized in that: obtaining a single-frame time-series image of text scanning; Perform image preprocessing on the sequence images to obtain a preprocessed scanned image; perform image splicing operations based on image pyramid template matching on the preprocessed scanned images of adjacent frames to obtain a spliced image; perform text recognition on the spliced image to obtain a text scanned image text recognition results. The text recognition method for text scanning images of the present invention is especially suitable for smart watches equipped with a special scanning camera for text scanning recognition, and solves the problem that the existing text recognition method is applied to smart watches equipped with a special scanning camera for text scanning recognition. Low recognition rate and poor user experience.

Description

Character recognition method for scanned character image, smart watch and electronic device

technical field

The invention relates to the field of smart mobile devices, in particular to a character recognition method for scanned character images, smart watches and electronic devices.

Background technique

In work and life, it is often necessary to scan and identify text in pictures. For example, when encountering some web pages that cannot directly copy text, or PPT and PDF documents, the efficiency of manually copying them is too low. With the help of text scanning and recognition technology, it can be greatly improved. Acquisition efficiency of text information in pictures.

With the development of social informatization, text scanning and recognition functions are more and more configured on various hardware and software products. For smart watches, their own functions have been expanded to realize text scanning and recognition functions. Existing smart watches realize the function of text scanning and recognition, and generally use their own camera to capture text objects. Due to the difference in hardware structure between the camera and the dedicated scanning camera, the recognition rate of text images captured by the camera will be different when performing character recognition. Reduced, the user experience of using smart watches for text scanning and recognition is not good.

In order to solve the text scanning recognition rate of the smart watch, the existing smart watch is combined with a scanning camera specially used for text scanning. When using the smart watch for text scanning recognition, the configured scanning camera is used for text scanning, which improves the text recognition rate. , and the user is more convenient to use. A successful case of the mature application of the scanning camera in mobile smart hardware is the intelligent scanning pen. By setting the scanning camera on the tip of the pen and keeping a fixed angle and distance from the scanned text object, the intelligent scanning pen can obtain a text object that is more suitable for being scanned. Image of scanned text with recognized text information. Smart watches are limited by the size of the watch body. Although the space can be optimized to install the scanning camera inside the watch body, the distance and angle between the scanning camera and the scanned text object are different from those of the smart scanning pen. Therefore, if the scanning camera is simply "plugged" into the body of the smart watch without optimizing the algorithm on the software, the integrated scanning camera of the smart watch cannot effectively improve the text scanning recognition rate.

Therefore, the technical problem to be solved by the present invention is: how to specifically optimize the software algorithm to improve the recognition rate of character recognition for a smart watch equipped with a special scanning camera for character scanning and recognition?

In view of this, the present invention is proposed.

Contents of the invention

In order to solve the above problems, the first object of the present invention is to provide a text recognition method adaptable to the text scanning image of a smart watch equipped with a special scanning camera for text scanning recognition. Specifically, the following technical solutions are adopted:

A character recognition method for a character scanned image, comprising:

Obtain a single-frame time-series image of text scanning;

performing image preprocessing on the single-frame time-series image to obtain a pre-processed scanned image;

performing an image stitching operation based on image pyramid template matching on the pre-processed scanned images of adjacent frames to obtain a stitched image;

Character recognition is performed on the spliced image to obtain a character recognition result of the scanned character image.

As an optional implementation mode of the present invention, in a method for character recognition of a character scanned image in this embodiment, the acquisition of a single-frame time-series image of the character scan includes:

Obtain n single-frame time-series images {P1,...,Pn} of text scanning, and image Pn represents the nth single-frame time-series image;

Perform image preprocessing on the single-frame time-series image, and obtain the pre-processed scanned image including:

Perform clipping processing on any single-frame time-series image Pn in the single-frame time-series images {P1,...,Pn}, and remove the target clipping part selected according to preset conditions in the single-frame time-series image Pn;

performing brightness compensation processing on the clipped single-frame time-series images Pn;

A tilt correction process is performed on the single-frame time-series image Pn that has undergone the brightness compensation process to obtain a pre-processed scanned image.

As an optional implementation mode of the present invention, in a character recognition method of a scanned character image in this embodiment, the tilt correction is performed on the single-frame time-series image Pn that has undergone the brightness compensation process, and the pre-processed scanned image obtained includes :

Sequentially performing tilt correction and perspective transformation on the single-frame time-series images Pn processed by the brightness compensation, the correction angle parameter of the tilt correction is a preset value α;

Perform Gamma compensation and dynamic linear compensation on the corrected image to obtain a uniform background brightness and obtain a pre-processed scanned image;

Preprocess the n single-frame time series images {P1,...,Pn} to obtain n preprocessed scanned images {newP1, newP2,...,newPn}.

As an optional implementation of the present invention, in the text recognition method of a scanned text image in this embodiment, the image stitching operation based on image pyramid template matching is performed on the pre-processed scanned images of adjacent frames, and the stitched image includes: :

Extract the sift feature from the current frame image newPn arriving in time sequence;

The sift feature map is sampled in a multi-layer image pyramid, and template matching is performed layer by layer with the image pyramid feature image of the previous frame image newPre, and the matching position of the upper image pyramid is used as a candidate parameter for the matching position of the lower layer.

Accelerate the matching process, where the previous frame image of the first frame image is the same as the current frame by default;

Copy and stitch the images whose positions have been matched to obtain a stitched image, output the stitched image and display it on the screen.

As an optional implementation mode of the present invention, in a method for character recognition of a scanned character image in this embodiment, performing character recognition on the spliced image to obtain a character recognition result of the scanned character image includes:

The n pieces of single-frame time-series images {P1,...,Pn} undergo an image stitching operation to obtain n pieces of spliced images {SP1,SP2,...,SPn};

Input the incremental part of the current spliced image SPn into the CNN network to obtain the incremental part dFS (FeatureSequence);

Input the incremental part dFS (Feature Sequence) of one or more frames of spliced image SPn into CRNN+CTC network for character sequence recognition, and obtain the character recognition result of the character scanned image.

As an optional implementation mode of the present invention, a text recognition method of a text scanning image in this embodiment includes:

preset increment preset max{dFS(Feature Sequence)};

When the incremental part dFS (Feature Sequence) of the stitched image of the previous frame or multiple frames reaches the preset incremental preset max{dFS (Feature Sequence)}, the incremental part dFS of the stitched image of the current frame or multiple frames (Feature Sequence) Enter the CRNN+CTC network for text sequence recognition.

As an optional implementation mode of the present invention, a text recognition method of a text scanning image in this embodiment includes:

Input the incremental part of the current spliced image SPn into the CNN network to obtain the dFS (Feature Sequence) of the incremental part, and judge whether the incremental part dFS (Feature Sequence) reaches the preset incremental preset max{dFS (Feature Sequence)} ;

If the judgment result is yes, then input the dFS (Feature Sequence) of the incremental part of the current spliced image SPn into the CRNN+CTC network for character sequence recognition; Merge the dFS (Feature Sequence) of the incremental part of the spliced image SPnPre that has not been input into the CRNN+CTC network in one frame or the previous few frames, and further judge whether it reaches the preset incremental preset max{dFS (Feature Sequence)};

If the judgment result is yes, merge the dFS (Feature Sequence) of the incremental part of the stitched image of the current spliced image SPn and the previous frame or previous frames into the CRNN+CTC network for text sequence recognition, if the judgment result is no , then continue to acquire the incremental part of the next frame of stitched image SPnext and merge it until the merged incremental part reaches max{dFS(Feature Sequence)}, or all stitched images are processed.

The second object of the present invention is to provide a smart watch, using the text recognition method of the text scanning image, while improving the hardware by using the scanning camera, by using the text recognition method of the text scanning image of the present invention. The optimization of the software algorithm can improve the recognition rate of the text scanning recognition function and improve the user experience. Specifically, the following technical solutions are adopted:

A smart watch, comprising:

The scanning module acquires single-frame time-series images of text scanning;

An image preprocessing module, which performs image preprocessing on the single-frame time series image to obtain a preprocessed scanned image;

The image stitching module performs an image stitching operation based on image pyramid template matching on the pre-processed scanned images of adjacent frames to obtain a stitched image;

and a character recognition module, which performs character recognition on the spliced image to obtain a character recognition result of the character scanned image.

The present invention also provides electronic equipment, including a processor and a memory, the memory is used to store a computer executable program, and when the computer program is executed by the processor, the processor executes the above-mentioned text scanning image text recognition method.

The present invention also provides a computer-readable storage medium storing a computer-executable program. When the computer-executable program is executed, the above-mentioned method for character recognition of a scanned character image is realized.

Compared with prior art, the beneficial effect of the present invention:

The text recognition method for scanned text images of the present invention performs image preprocessing on scanned single-frame time-series images, and the pre-processed scanned images are then subjected to subsequent image splicing and text recognition. In this way, even if the scanned single-frame time-series images exist Defects and problems can also improve the quality of scanned images through image preprocessing, thereby improving the recognition rate of text recognition. It is especially suitable for smart watches equipped with special scanning cameras for text scanning and recognition, and solves the application of existing text recognition methods. In smart watches equipped with special scanning cameras for text scanning and recognition, the text scanning recognition rate is low and the user experience is poor.

The character recognition method of the scanned character image of the present invention performs the image stitching operation based on image pyramid template matching on the preprocessed scanned images of adjacent frames, which can accelerate the matching process, increase the speed of image stitching, and accelerate the obtaining of character recognition results.

Description of drawings:

Fig. 1 is the flow chart of the text recognition method of the text scanning image of the embodiment of the present invention;

Figure 2 is a flow chart of text scanning and recognition of smart watches according to an embodiment of the present invention;

Fig. 3 is a schematic diagram of a three-dimensional structure of a smart watch according to an embodiment of the present invention;

Fig. 4 is a schematic diagram of the three-dimensional structure of the watch body of the smart watch according to the embodiment of the present invention.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Apparently, the described embodiments are some, not all, embodiments of the present invention.

Therefore, the following detailed description of the embodiments of the present invention is not intended to limit the scope of the claimed invention, but merely represents some embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

It should be noted that, in the case of no conflict, the embodiments of the present invention and the features and technical solutions in the embodiments can be combined with each other.

It should be noted that like numerals and letters denote similar items in the following figures, therefore, once an item is defined in one figure, it does not require further definition and explanation in subsequent figures.

In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "upper", "lower", etc. is based on the orientation or positional relationship shown in the drawings, or the conventionally placed position when the product of the invention is used. Orientation or positional relationship, or the orientation or positional relationship commonly understood by those skilled in the art, such terms are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the referred device or element must have a specific orientation, Constructed and operative in a particular orientation and therefore are not to be construed as limitations of the invention. In addition, the terms "first", "second", etc. are only used for distinguishing descriptions, and should not be construed as indicating or implying relative importance.

Referring to shown in Fig. 1, a kind of text recognition method of text scanning image of the present embodiment comprises:

Obtain a single-frame time-series image of text scanning;

performing image preprocessing on the single-frame time-series image to obtain a pre-processed scanned image;

performing an image stitching operation based on image pyramid template matching on the pre-processed scanned images of adjacent frames to obtain a stitched image;

Character recognition is performed on the spliced image to obtain a character recognition result of the scanned character image.

The text recognition method for scanned text images in this embodiment performs image preprocessing on scanned single-frame time-series images, and the pre-processed scanned images are then subjected to subsequent image splicing and text recognition. In this way, even scanned single-frame time-series images There are defects and problems, and the quality of scanned images can also be improved through image preprocessing, thereby improving the recognition rate of text recognition. It is especially suitable for smart watches equipped with special scanning cameras for text scanning and recognition, and solves the existing text recognition methods. Applied to smart watches equipped with a special scanning camera for text scanning and recognition, the text scanning recognition rate is low, and the user experience is poor.

The text recognition method for scanned text images of this embodiment performs image stitching operations based on image pyramid template matching on preprocessed scanned images of adjacent frames, which can speed up the matching process, increase the speed of image stitching, and speed up obtaining text recognition results.

Further, in a method for character recognition of scanned character images in this embodiment, the acquisition of single-frame time-series images of character scans includes:

Acquire n single-frame time-series images {P1,...,Pn} of text scanning, and image Pn represents the nth single-frame time-series image.

Perform image preprocessing on the single-frame time-series image, and obtain the pre-processed scanned image including:

Perform clipping processing on any single-frame time-series image Pn in the single-frame time-series images {P1,...,Pn}, and remove the target clipping part selected according to preset conditions in the single-frame time-series image Pn, specifically, Unnecessary and covered parts in the single-frame time series image Pn can be removed through preset conditions, and the unnecessary parts include, for example, set graphics, pictures, logos, etc.;

performing brightness compensation processing on the clipped single-frame time-series images Pn;

A tilt correction process is performed on the single-frame time-series image Pn that has undergone the brightness compensation process to obtain a pre-processed scanned image.

Further, in a text recognition method for scanned text images implemented in this city, the tilt correction is performed on the single-frame time-series images Pn that have undergone the brightness compensation processing, and the pre-processed scanned images obtained include:

Sequentially performing tilt correction and perspective transformation on the single-frame time-series images Pn processed by the brightness compensation, the correction angle parameter of the tilt correction is a preset value α;

Perform Gamma compensation and dynamic linear compensation on the corrected image to obtain a uniform background brightness and obtain a pre-processed scanned image;

Preprocess the n single-frame time series images {P1,...,Pn} to obtain n preprocessed scanned images {newP1, newP2,...,newPn}.

Specifically, the preset value α of the correction angle parameter of the tilt correction in this embodiment needs to consider the tilt angle of the user relative to the scanned text object when using the smart watch to scan the image, which is determined by a large number of image experiments. Specifically, the The preset value α of the correction angle parameter can be selected as 65°.

The compensation parameters in this embodiment are obtained from a large number of image experiments. Specifically, the Gamma coefficient used is -0.3, and the dynamic linear compensation coefficient is 330.

Further, in the text recognition method of a text scanned image of the present embodiment, performing an image stitching operation based on image pyramid template matching on the pre-processed scanned images of adjacent frames to obtain a stitched image includes:

Extract the sift feature from the current frame image newPn arriving in time sequence;

The sift feature map is sampled in a multi-layer image pyramid, and template matching is performed layer by layer with the image pyramid feature image of the previous frame image newPre, and the matching position of the upper image pyramid is used as a candidate parameter for the matching position of the lower layer.

Accelerate the matching process, where the previous frame image of the first frame image is the same as the current frame by default;

Copy and stitch the images whose positions have been matched to obtain a stitched image, output the stitched image and display it on the screen.

Specifically, the sift feature map is sampled in a three-layer image pyramid, and template matching is performed layer by layer with the image pyramid feature image of the previous frame image newPre, and the matching position of the upper pyramid is used as a candidate parameter for the matching position of the lower layer to speed up the matching process. In addition, the previous frame of the first frame image is the same as the current frame by default.

In addition, the mosaic image in this embodiment will be displayed on the screen of the smart watch. During the scanning movement of the smart watch, when the time-series image newP arrives continuously, the images will continue to be spliced and grow, forming a continuously growing pattern on the screen with the scanning movement. dynamic picture.

Further, in the character recognition method of the scanned character image in this embodiment, performing character recognition on the spliced image to obtain the character recognition result of the scanned character image includes:

The n pieces of single-frame time-series images {P1,...,Pn} undergo an image stitching operation to obtain n pieces of spliced images {SP1,SP2,...,SPn};

Input the incremental part of the current spliced image SPn into the CNN network to obtain the incremental part dFS (FeatureSequence);

Input the incremental part dFS (Feature Sequence) of one or more frames of spliced image SPn into CRNN+CTC network for character sequence recognition, and obtain the character recognition result of the character scanned image.

Specifically, a character recognition method of a character scanned image in this embodiment includes:

preset increment preset max{dFS(Feature Sequence)};

When the incremental part dFS (Feature Sequence) of the stitched image of the previous frame or multiple frames reaches the preset incremental preset max{dFS (Feature Sequence)}, the incremental part dFS of the stitched image of the current frame or multiple frames (Feature Sequence) Enter the CRNN+CTC network for text sequence recognition. Incremental recognition greatly speeds up text recognition, and an incremental threshold max{dFS(Feature Sequence)} (experience value is 28) can be set to reduce the number of CRNN+CTC network startups. After the scan is completed, when the watch is lifted, as long as the increment is > 1, the increment recognition will be started again at last.

Specifically, a character recognition method of a character scanned image in this embodiment includes:

Input the incremental part of the current spliced image SPn into the CNN network to obtain the dFS (Feature Sequence) of the incremental part, and judge whether the incremental part dFS (Feature Sequence) reaches the preset incremental preset max{dFS (Feature Sequence)} ;

If the judgment result is yes, then input the dFS (Feature Sequence) of the incremental part of the current spliced image SPn into the CRNN+CTC network for character sequence recognition; Merge the dFS (Feature Sequence) of the incremental part of the spliced image SPnPre that has not been input into the CRNN+CTC network in one frame or the previous few frames, and further judge whether it reaches the preset incremental preset max{dFS (Feature Sequence)};

If the judgment result is yes, merge the dFS (Feature Sequence) of the incremental part of the stitched image of the current spliced image SPn and the previous frame or previous frames into the CRNN+CTC network for text sequence recognition, if the judgment result is no , then continue to acquire the incremental part of the next frame of stitched image SPnext and merge it until the merged incremental part reaches max{dFS(Feature Sequence)}, or all stitched images are processed.

This embodiment also provides a smart watch, including:

The scanning module acquires single-frame time-series images of text scanning;

An image preprocessing module, which performs image preprocessing on the single-frame time series image to obtain a preprocessed scanned image;

The image stitching module performs an image stitching operation based on image pyramid template matching on the pre-processed scanned images of adjacent frames to obtain a stitched image;

and a character recognition module, which performs character recognition on the spliced image to obtain a character recognition result of the character scanned image.

The text scanning module 300 of the smart watch of this embodiment is specially used for character scanning and recognition, and the characters include letters, numbers, arithmetic symbols, punctuation marks and other symbols, as well as some functional symbols; the text scanning module 300 generally uses a fixed-focus scanning camera, The field of view FOV of the scanning camera is generally large to obtain relatively complete scanned characters. With the supplementary light source with set brightness, the text scanning module 300 needs to maintain a set distance and angle from the scanned object when in use, and cooperate with the supplementary light. With the supplementary light of the light source, the camera can capture images that are more suitable for character recognition, improving the accuracy of character scanning and recognition.

The text scanning module 300 of the smart watch in this embodiment is directly arranged on the watch body 100, which is an innovative structural design in the smart watch industry. Setting the text scanning module 300 on the watch body 100 is easier for the integrated installation of functional modules , and it is necessary to carry out a comprehensive waterproof structure design for the watch body 100, the user can perform various functional operations on the watch body 100 during use, which is convenient for the user to operate and use.

The smart watch of this embodiment realizes the text scanning recognition function of the smart watch by configuring the text scanning module 300 specially used for text scanning. The text recognition method of the text scanning image adopted by the smart watch of this embodiment is aimed at scanning single Image preprocessing is performed on the time series images, and the preprocessed scanned images are then subjected to subsequent image stitching and text recognition. In this way, even if there are defects and problems in the scanned single frame time series images, the scanned images can be improved through image preprocessing quality, thereby improving the recognition rate of text recognition, especially suitable for smart watches equipped with special scanning cameras for text scanning and recognition, solving the existing text recognition methods applied to smart watches equipped with special scanning cameras for text scanning and recognition, text Low scanning recognition rate and poor user experience.

Refer to FIG. 2 for the flow chart of character scanning and recognition performed by the smart watch of this embodiment.

Further, the scanning module of the smart watch in this embodiment acquires a single-frame time-series image process of text scanning including:

Acquire n single-frame time-series images {P1,...,Pn} of text scanning, and image Pn represents the nth single-frame time-series image.

The image preprocessing module of the smart watch in this embodiment performs image preprocessing on the single-frame time-series image, and the obtained pre-processed scanned image includes:

Perform clipping processing on any single-frame time-series image Pn in the single-frame time-series images {P1,...,Pn}, and remove the target clipping part selected according to preset conditions in the single-frame time-series image Pn;

performing brightness compensation processing on the clipped single-frame time-series images Pn;

A tilt correction process is performed on the single-frame time-series image Pn that has undergone the brightness compensation process to obtain a pre-processed scanned image.

The image preprocessing module of the smart watch in this embodiment performs tilt correction on the single-frame time-series images Pn processed by the brightness compensation, and obtains the pre-processed scanned images including:

Sequentially performing tilt correction and perspective transformation on the single-frame time-series images Pn processed by the brightness compensation, the correction angle parameter of the tilt correction is a preset value α;

Perform Gamma compensation and dynamic linear compensation on the corrected image to obtain a uniform background brightness and obtain a pre-processed scanned image;

Preprocess the n single-frame time series images {P1,...,Pn} to obtain n preprocessed scanned images {newP1, newP2,...,newPn}.

The image stitching module of the smart watch of this embodiment performs image stitching operations based on image pyramid template matching on the pre-processed scanned images of adjacent frames, and the stitched images obtained include:

Extract the sift feature from the current frame image newPn arriving in time sequence;

The sift feature map is sampled in a multi-layer image pyramid, and template matching is performed layer by layer with the image pyramid feature image of the previous frame image newPre, and the matching position of the upper image pyramid is used as a candidate parameter for the matching position of the lower layer.

Accelerate the matching process, where the previous frame image of the first frame image is the same as the current frame by default;

Copy and stitch the images whose positions have been matched to obtain a stitched image, output the stitched image and display it on the screen.

The text recognition module of the smart watch of this embodiment performs text recognition on the spliced image, and the text recognition results obtained from the text scanning image include:

The n pieces of single-frame time-series images {P1,...,Pn} undergo an image stitching operation to obtain n pieces of spliced images {SP1,SP2,...,SPn};

Input the incremental part of the current spliced image SPn into the CNN network to obtain the incremental part dFS (FeatureSequence);

Input the incremental part dFS (Feature Sequence) of one or more frames of spliced image SPn into CRNN+CTC network for character sequence recognition, and obtain the character recognition result of the character scanned image.

The text recognition module of the smart watch of this embodiment performs text recognition on the spliced image, and the text recognition results obtained from the text scanning image include:

preset increment preset max{dFS(Feature Sequence)};

When the incremental part dFS (Feature Sequence) of the stitched image of the previous frame or multiple frames reaches the preset incremental preset max{dFS (Feature Sequence)}, the incremental part dFS of the stitched image of the current frame or multiple frames (Feature Sequence) Enter the CRNN+CTC network for text sequence recognition.

The text recognition module of the smart watch of this embodiment performs text recognition on the spliced image, and the text recognition results obtained from the text scanning image include:

Input the incremental part of the current spliced image SPn into the CNN network to obtain the dFS (Feature Sequence) of the incremental part, and judge whether the incremental part dFS (Feature Sequence) reaches the preset incremental preset max{dFS (Feature Sequence)} ;

If the judgment result is yes, then input the dFS (Feature Sequence) of the incremental part of the current spliced image SPn into the CRNN+CTC network for character sequence recognition; Merge the dFS (Feature Sequence) of the incremental part of the spliced image SPnPre that has not been input into the CRNN+CTC network in one frame or the previous few frames, and further judge whether it reaches the preset incremental preset max{dFS (Feature Sequence)};

If the judgment result is yes, merge the dFS (Feature Sequence) of the incremental part of the stitched image of the current spliced image SPn and the previous frame or previous frames into the CRNN+CTC network for text sequence recognition, if the judgment result is no , then continue to acquire the incremental part of the next frame of stitched image SPnext and merge it until the merged incremental part reaches max{dFS(Feature Sequence)}, or all stitched images are processed.

Referring to Figure 3 and Figure 4, since the smart watch of this embodiment uses a special character scanning module 300 to scan characters, in order to ensure the character scanning recognition effect of the character scanning module 300, it is necessary to improve the structure of the smart watch to adapt to the Describe the text scanning module, specifically, this embodiment implements the control for the text scanning module 300 through the combination of software and hardware, by obtaining the distance and angle between the text scanning module 300 and the scanned object, thereby judging whether it satisfies The condition that the text scanning module 300 scans, if the judgment result is yes, then the text scanning module 300 is controlled to scan, if the judgment result is no, then the text scanning module 300 does not scan, when the smart watch runs the scanning program, but the text scanning module 300 does not execute the scanning operation, it is necessary to prompt information on the screen 101 of the smart watch, and also provide operation guidance to the user, so as to realize the text scanning module 300 to scan.

In the smart watch disclosed in this embodiment, a character scanning module 300 specially used for character scanning is arranged on the watch body 100, and the character scanning and recognition can be realized through the character scanning module 300, and the characters recognized by scanning can be directly displayed synchronously. Simultaneous translation can be performed on the scanned and recognized text to realize the text scanning and translation function.

Specifically, in order to facilitate the user to use the character scanning module 300 to scan characters, the watch body 100 described in this embodiment is provided with a scanning positioning part 400, and the scanning positioning part 400 is aligned with the scanned object when the character scanning module 300 scans. contact, so that the text scanning module 300 maintains a set distance and angle with the object to be scanned.

As an optional implementation of this embodiment, the scanning positioning part 400 described in this embodiment is a positioning bump disposed on the watch body 100 close to the character scanning module 300, and the positioning bump has an inclined positioning surface. The inclined positioning surface is in close contact with the scanned object so that the character scanning module 300 maintains a preset distance and a preset tilt angle from the scanned object, and the character scanning module 300 satisfies the preset scanning conditions and can scan characters.

Further, in order to judge whether the scanning positioning part 400 is in the correct scanning positioning position, a detection sensor for monitoring the position of the scanning positioning part 400 is arranged on the watch body described in this embodiment, and the detection sensor is in the When the scanning positioning part is in contact with the object to be scanned, a detection signal is triggered, and the main control module of the smart watch triggers the character scanning module to scan according to the detection signal.

As an optional implementation of this embodiment, the detection sensor in this embodiment is a distance sensor or a push switch arranged on the inclined positioning surface, and the distance sensor can detect the distance between the inclined positioning surface and the object to be scanned. In order to judge whether the two are attached, the push switch is closed when the inclined positioning surface is in contact with the object to be scanned, and an electric signal is sent.

When the smart watch of this embodiment uses the character scanning function, it first triggers and starts the character scanning module. After the character scanning module is triggered and starts, it does not start scanning directly; further, it is necessary to determine the current smart watch and the scanned object. Whether the distance and angle between them meet the conditions for the text scanning module to perform scanning work. Specifically, by judging whether the scanning positioning part is at the positioning position of the scanned object, if the scanning positioning part is at the positioning position of the scanned object, it will directly trigger the text scanning module. The supplementary light is turned on, and the scanning camera starts to scan the image. If the scanning positioning part is not in the positioning position of the scanned object, the text scanning module is controlled to be in a standby state, and the scanning is not performed, and information is displayed on the screen of the smart watch at the same time. As a reminder, the user can also be guided to operate, so as to realize the text scanning module to scan.

As an optional implementation of this embodiment, the smart watch of this embodiment includes:

The trigger module is arranged on the watch body, and the trigger module is connected to the main control module in communication. After the scanning recognition program is started, the trigger module monitors the distance between the scanned object and the object to be scanned. When the distance is detected When the value is less than the preset value, the text scanning module is controlled to be turned on to acquire scanned pictures.

As an optional implementation of this embodiment, the smart watch described in this embodiment includes a strap assembly 200, the watch body 100 and the strap assembly 200 are actively connected, and the smart watch has:

In the first state, the watch body 100 and the strap assembly 200 are mutually locked for wearing a smart watch;

In the second state, the watch body 100 and the strap assembly 200 are unlocked, and the watch body 100 can move relative to the strap assembly 200 to a preset position for scanning by the text scanning module 300 to scan character information.

In this embodiment, the user can adjust the relative state between the watch body 100 and the strap assembly 200 according to the use requirements by adopting a flexible connection between the watch body 100 and the strap assembly 200, thereby greatly facilitating the user to scan characters operate.

Further, the watch body 100 and the strap assembly 200 are detachably connected, and the smart watch has:

In the first state, the watch body 100 is fixedly connected with the strap assembly 200 for wearing a smart watch;

In the second state, the watch body 100 is disassembled and separated from the strap assembly 200 for the character scanning module to scan character information.

In this embodiment, by adopting a detachable connection between the watch body 100 and the strap assembly 200, after the user wears the smart watch on his wrist, if he wants to use the text scanning module 300 to scan characters, he can directly connect the watch body 100 is disassembled from the strap assembly 200 for scanning operation, and after the scanning is completed, the watch body can be put back on the strap assembly.

Further, there is a first state and a second state between the watch body 100 and the strap assembly 200 of this embodiment, and the second state is mainly used to realize the character scanning of the scanning assembly, therefore, this embodiment can judge the Whether the state of the watch body 100 relative to the strap assembly 200 is judged to automatically trigger the opening of the text scanning module, the specific scheme is as follows:

Specifically, the trigger module of the smart watch of this embodiment is set on the watch body 100, and the trigger module is configured to: monitor and determine when the watch body 100 is in the second state relative to the strap assembly 200 and automatically generate a trigger instruction to control the character scanning module to scan character information.

Further, in order to realize the application of smart watch image collection, such as taking pictures, videos, video calls and face recognition, etc., the smart watch of this embodiment includes a camera module 500, and the camera module 500 is set on the watch On the body 100, the camera module 500 is configured to perform image capture applications, and the image capture applications include taking pictures or videos, video calling and face recognition.

This embodiment also provides a computer-readable storage medium storing a computer-executable program. When the computer-executable program is executed, the character recognition method of the scanned character image is implemented.

The storage medium in this embodiment may include a data signal propagated in baseband or as part of a carrier wave, and readable program codes are carried therein. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A readable storage medium may also be any readable medium other than a readable storage medium that can send, propagate or transport a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the readable storage medium may be transmitted by any suitable medium, including but not limited to wireless, cable, optical cable, RF, etc., or any suitable combination of the above.

This embodiment also provides an electronic device, including a processor and a memory, the memory is used to store a computer executable program, and when the computer program is executed by the processor, the processor executes the text scanning Text recognition methods for images.

Electronic devices take the form of general-purpose computing devices. There can be one or more processors and they work together. The present invention does not exclude distributed processing, that is, processors may be dispersed in different physical devices. The electronic device of the present invention is not limited to a single entity, but may also be the sum of multiple entity devices.

The memory stores a computer executable program, typically machine readable code. The computer-readable program can be executed by the processor, so that the electronic device can execute the method of the present invention, or at least part of the steps in the method.

The memory includes a volatile memory, such as a random access memory unit (RAM) and/or a cache memory unit, and may also be a nonvolatile memory, such as a read only memory unit (ROM).

It should be understood that the electronic device of the present invention may also include elements or components not shown in the above examples. For example, some electronic devices also include display units such as display screens, and some electronic devices also include human-computer interaction components, such as buttons and keyboards. As long as the electronic device can execute the computer-readable program in the memory to realize the method or at least some steps of the method of the present invention, it can be considered as the electronic device covered by the present invention.

Through the above description of the embodiments, those skilled in the art can easily understand that the present invention can be realized by hardware capable of executing specific computer programs, such as the system of the present invention, and the electronic processing unit, server, client, Mobile phones, control units, processors, etc. The present invention can also be implemented by computer software that executes the method of the present invention, such as by control software executed by a microprocessor, an electronic control unit, a client, a server, and the like. However, it should be noted that the computer software for executing the method of the present invention is not limited to be executed by one or a specific hardware entity, and it may also be implemented in a distributed manner by unspecified specific hardware. For computer software, the software product can be stored in a computer-readable storage medium (which can be CD-ROM, U disk, mobile hard disk, etc.), or can be distributed and stored on the network, as long as it can make the electronic equipment execute according to this document. method of invention.

The above embodiments are only used to illustrate the present invention and are not intended to limit the technical solutions described in the present invention. Although the specification has described the present invention in detail with reference to the above-mentioned embodiments, the present invention is not limited to the above-mentioned specific implementation methods, so Any modifications or equivalent replacements to the present invention; and all technical solutions and improvements that do not depart from the spirit and scope of the invention are covered by the scope of the claims of the present invention.

Claims (10)

1. A text recognition method of text scanning image, is characterized in that, comprises: Obtain a single-frame time-series image of text scanning; performing image preprocessing on the single-frame time-series image to obtain a pre-processed scanned image; performing an image stitching operation based on image pyramid template matching on the pre-processed scanned images of adjacent frames to obtain a stitched image; Character recognition is performed on the spliced image to obtain a character recognition result of the scanned character image. 2. the text recognition method of a kind of text scanning image according to claim 1, is characterized in that, the single-frame time series image of described acquisition text scanning comprises: Obtain n single-frame time-series images {P1,...,Pn} of text scanning, and image Pn represents the nth single-frame time-series image; Perform image preprocessing on the single-frame time-series image, and obtain the pre-processed scanned image including: Perform clipping processing on any single-frame time-series image Pn in the single-frame time-series images {P1,...,Pn}, and remove the target clipping part selected according to preset conditions in the single-frame time-series image Pn; performing brightness compensation processing on the clipped single-frame time-series images Pn; A tilt correction process is performed on the single-frame time-series image Pn that has undergone the brightness compensation process to obtain a pre-processed scanned image. 3. The character recognition method of a scanned character image according to claim 2, wherein said performing tilt correction on the single-frame time-series image Pn processed through said brightness compensation to obtain a pre-processed scanned image comprises: Sequentially performing tilt correction and perspective transformation on the single-frame time-series images Pn processed by the brightness compensation, the correction angle parameter of the tilt correction is a preset value α; Perform Gamma compensation and dynamic linear compensation on the corrected image to obtain a uniform background brightness and obtain a pre-processed scanned image; Preprocess the n single-frame time series images {P1,...,Pn} to obtain n preprocessed scanned images {newP1, newP2,...,newPn}. 4. the text recognition method of a kind of text scanning image according to claim 3, it is characterized in that, described preprocessing scanning image of adjacent frame is carried out the image splicing operation based on image pyramid template matching, obtains splicing image and comprises: Extract the sift feature from the current frame image newPn arriving in time sequence; The sift feature map is sampled in a multi-layer image pyramid, and template matching is performed layer by layer with the image pyramid feature image of the previous frame image newPre, and the matching position of the upper image pyramid is used as a candidate parameter for the matching position of the lower layer. Accelerate the matching process, where the previous frame image of the first frame image is the same as the current frame by default; Copy and stitch the images whose positions have been matched to obtain a stitched image, output the stitched image and display it on the screen. 5. the text recognition method of a kind of text scanning image according to claim 4, it is characterized in that, described character recognition is carried out for described mosaic image, the text recognition result that obtains text scanning image comprises: The n pieces of single-frame time-series images {P1,...,Pn} undergo an image stitching operation to obtain n pieces of spliced images {SP1,SP2,...,SPn}; Input the incremental part of the current spliced image SPn into the CNN network to obtain the incremental part dFS (Feature Sequence); Input the incremental part dFS (Feature Sequence) of one or more frames of spliced image SPn into CRNN+CTC network for character sequence recognition, and obtain the character recognition result of the character scanned image. 6. The text recognition method of a kind of text scanning image according to claim 5, is characterized in that, comprises: preset increment preset max{dFS(Feature Sequence)}; When the incremental part dFS (Feature Sequence) of the stitched image of the previous frame or multiple frames reaches the preset incremental preset max{dFS (Feature Sequence)}, the incremental part dFS of the stitched image of the current frame or multiple frames (Feature Sequence) Enter the CRNN+CTC network for text sequence recognition. 7. The character recognition method of a kind of character scan image according to claim 6, is characterized in that, comprises: Input the incremental part of the current spliced image SPn into the CNN network to obtain the dFS (Feature Sequence) of the incremental part, and judge whether the incremental part dFS (Feature Sequence) reaches the preset incremental preset max{dFS (Feature Sequence)} ; If the judgment result is yes, then input the dFS (Feature Sequence) of the incremental part of the current spliced image SPn into the CRNN+CTC network for character sequence recognition; Merge the dFS (Feature Sequence) of the incremental part of the spliced image SPnPre that has not been input into the CRNN+CTC network in one frame or the previous few frames, and further judge whether it reaches the preset incremental preset max{dFS (Feature Sequence)}; If the judgment result is yes, merge the dFS (Feature Sequence) of the incremental part of the stitched image of the current spliced image SPn and the previous frame or previous frames into the CRNN+CTC network for text sequence recognition, if the judgment result is no , then continue to acquire the incremental part of the next frame of stitched image SPnext and merge it until the merged incremental part reaches max{dFS(Feature Sequence)}, or all stitched images are processed. 8. A smart watch, characterized in that it comprises: The scanning module acquires single-frame time-series images of text scanning; An image preprocessing module, which performs image preprocessing on the single-frame time series image to obtain a preprocessed scanned image; The image stitching module performs an image stitching operation based on image pyramid template matching on the pre-processed scanned images of adjacent frames to obtain a stitched image; and a character recognition module, which performs character recognition on the spliced image to obtain a character recognition result of the character scanned image. 9. Electronic equipment, comprising a processor and a memory for storing computer-executable programs, characterized in that: When the computer program is executed by the processor, the processor executes a character recognition method for a character scanned image according to any one of claims 1-7. 10. A computer-readable storage medium, storing a computer-executable program, characterized in that, when the computer-executable program is executed, the text scanning image according to any one of claims 1-7 is realized. Text recognition method.

Images