CN115953789A - Adaptive text detection method, device, equipment and medium - Google Patents

Abstract

The embodiment of the present application provides an adaptive text detection method, device, equipment, and medium. The method includes inputting a text image into a text detection network; extracting feature information of a text area from a text image through a text detection network and obtaining a feature image; The image detects the boundary of the text area, and multiple corner points are used to locate the boundary to generate the area candidate frame of the text area; the adaptive threshold map is obtained from the feature image, and the adaptive threshold map is segmented to segment different texts belonging to the same text area To the same segmented area, the segmentation map is obtained; the text detection result is obtained by combining the area candidate map and the segmentation map; it can recognize and detect the image text of arbitrary shape and dense area, and improve the robustness of text detection.

Description

Adaptive text detection method, device, equipment and medium

technical field

The embodiments of the present application relate to, but are not limited to, the technical field of image processing, and in particular, relate to an adaptive text detection method, device, equipment, and medium.

Background technique

The text detection task is to determine the text position of the image and recognize the text expression of the image. In natural scene images, text is diverse, and the results of text detection tasks are affected by the size, font, orientation, scale, and shape of the text, as well as complex backgrounds and interference, such as uneven brightness, blur, low resolution, etc. Influence; these factors will lead to inaccurate results of the text detection task.

For the current text detection task, by giving a rotating rectangular frame, which contains 4 fixed corner points (upper left corner, upper right corner, lower right corner, lower left corner), represented by two-dimensional coordinates in the clockwise direction, this It is not suitable for the detection of arbitrary-shaped text; in addition, when the text areas are very close, the segmentation of dense text areas tends to fail, and only one of them can be predicted, resulting in missed detection. In the face of relatively dense text areas and scene images with large changes in text bending scales, the robustness of current text detection methods is difficult to guarantee.

Contents of the invention

The following is an overview of the topics described in detail in this article. This summary is not intended to limit the scope of the claims.

The embodiment of the present application aims to solve at least one of the technical problems existing in the prior art. The embodiment of the present application provides an adaptive character detection method, device, equipment and medium, which can recognize and detect image characters in arbitrary shapes and dense areas , to improve the robustness of text detection.

In an embodiment of the first aspect of the present application, an adaptive text detection method includes:

Obtain a text image to be detected, and input the text image to a text detection network;

Extracting feature information of the text area from the text image, and obtaining a feature image according to the feature information;

Perform boundary detection processing on the feature image to obtain the boundary of the text region, locate the boundary of the text region through a plurality of corner points, generate a region candidate frame for the character region, and obtain a region candidate map, the The number of corner points corresponds to the proportion of the text area;

Carrying out adaptive thresholding processing on the feature image to obtain an adaptive threshold map, performing image segmentation processing on the adaptive threshold map, and segmenting different characters belonging to the same text area into the same segmented area to obtain a segmentation map;

Combining the region candidate map and the segmentation map to perform text detection to obtain a text detection result.

In some embodiments of the first aspect of the present application, the feature information of the text area is extracted from the text image, and the feature image is obtained according to the feature information, including:

A plurality of residual blocks of the backbone network of the text detection network generate a plurality of basic feature maps containing feature information of text regions according to the text image;

Generate a plurality of fusion feature maps of different scales according to a plurality of the basic feature maps by the feature pyramid network of the text detection network;

A feature image is obtained by performing a concatenation operation and an upsampling operation on the plurality of fused feature maps.

In some embodiments of the first aspect of the present application, the fusion feature map includes a first feature map, a second feature map, a third feature map, a fourth feature map, and a fifth feature map; the first feature map, The sizes of the second feature map, the third feature map, the fourth feature map, and the fifth feature map decrease sequentially; the first feature map, the second feature map, the first feature map The number of channels of the three feature maps, the fourth feature map and the fifth feature map is the same.

In some embodiments of the first aspect of the present application, the performing a connection operation and an upsampling operation on a plurality of the fusion feature maps to obtain a feature image includes:

The first feature map, the result of the upsampling operation on the second feature map, the result of the upsampling operation on the third feature map, the result of the upsampling operation on the fourth feature map, and performing a concatenation operation on the results of the up-sampling operation on the fifth feature map to obtain a feature image.

In some embodiments of the first aspect of the present application, the boundary detection processing is performed on the feature image to obtain the boundary of the text area, and the boundary of the text area is located through a plurality of corner points to generate the Area candidate boxes for text areas, including:

Perform boundary detection processing on the feature image, and represent the boundary of the character area in the feature image through the initial corner point;

calculating offsets of different corner points, performing curvature prediction according to the offsets, and obtaining the curvature of the text area;

The corner points are increased or decreased according to the curvature to locate the boundaries representing the text regions in the feature image, and then the borders of the text regions are positioned through the plurality of corner points to generate region candidate frames for the text regions.

In some embodiments of the first aspect of the present application, the adaptive thresholding process is performed on the feature image to obtain an adaptive threshold value map, and the image segmentation process is performed on the adaptive threshold value map, and the images belonging to the same text area Different characters are segmented into the same segmented area to obtain a segmented graph, including:

calculating pixel values for the feature image, and performing adaptive thresholding processing according to the pixel values to obtain an adaptive threshold map;

calculating a feature distance between characters according to the pixel values, and segmenting different characters belonging to the same character area into the same segmented area according to the feature distance, so as to perform image segmentation processing on the adaptive threshold map to obtain a segmented map.

In some embodiments of the first aspect of the present application, the thresholds at different positions of the text area in the adaptive threshold map are different, and the threshold at the border of the text area is smaller than the threshold at the center of the text area.

In an embodiment of the second aspect of the present application, an adaptive text detection device includes:

The input module is used to obtain the text image to be detected, and input the text image to the text detection network;

A feature extraction module, configured to extract feature information of a text region from the text image, and obtain a feature image according to the feature information;

An adaptive corner detection module, configured to perform boundary detection processing on the feature image to obtain the boundary of the text area, and locate the boundary of the text area through a plurality of corner points to generate an area candidate for the text area frame, the number of corner points corresponds to the proportion of the text area;

The adaptive threshold segmentation module is used to perform adaptive threshold processing on the feature image to obtain an adaptive threshold map, perform image segmentation processing on the adaptive threshold map, and segment different characters belonging to the same text area into the same segment region, get the segmentation map;

An output module, configured to combine the region candidate map and the segmentation map to perform text detection to obtain a text detection result.

According to the embodiment of the third aspect of the present application, an electronic device includes: a memory, a processor, and a computer program stored on the memory and operable on the processor. When the processor executes the computer program, the above-mentioned The adaptive text detection method described above.

In an embodiment of the fourth aspect of the present application, a computer-readable storage medium, the computer-readable storage medium is characterized in that the computer-readable storage medium stores one or more programs, and the one Or multiple programs can be run by one or more processors to implement the above-mentioned adaptive text detection method.

The above scheme has at least the following beneficial effects: Combining the two ideas based on regression and segmentation methods, an adaptive corner detection method and an adaptive threshold segmentation method are proposed to generate different numbers of corner points for text of different scales for positioning. The text segmentation map is generated by adapting the threshold method, and combined with the generated corner candidate boxes for joint optimization to obtain visual text detection results; it can recognize and detect image text in arbitrary shapes and dense areas, and improve the robustness of text detection.

Description of drawings

The accompanying drawings are used to provide a further understanding of the technical solution of the present application, and constitute a part of the specification, and are used together with the embodiments of the present application to explain the technical solution of the present application, and do not constitute a limitation to the technical solution of the present application.

Fig. 1 is a step diagram of the adaptive text detection method provided by the embodiment of the present application;

Fig. 2 is a substep diagram of step S200;

Fig. 3 is a sub-step diagram of step S300;

Fig. 4 is a sub-step diagram of step S400;

FIG. 5 is a structural diagram of an adaptive text detection device provided by an embodiment of the present application;

Fig. 6 is a structural diagram of an electronic device provided by an embodiment of the present application.

Detailed ways

In order to make the purpose, technical solution and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present application, not to limit the present application.

It should be noted that although the functional modules are divided in the schematic diagram of the device, and the logical sequence is shown in the flowchart, in some cases, it can be executed in a different order than the module division in the device or the flowchart in the flowchart. steps shown or described. The terms "first", "second" and the like in the specification, claims or the above drawings are used to distinguish similar objects, and not necessarily used to describe a specific order or sequence.

The embodiments of the present application will be further described below in conjunction with the accompanying drawings.

Embodiments of the present application provide an adaptive text detection method.

Referring to Figure 1, the adaptive text detection method includes:

Step S100, acquiring the text image to be detected, and inputting the text image into the text detection network;

Step S200, extracting feature information of the text area from the text image, and obtaining a feature image according to the feature information;

Step S300, performing boundary detection processing on the feature image to obtain the boundary of the text area, and locating the boundary of the text area through multiple corner points to generate an area candidate box for the text area;

Step S400, performing adaptive thresholding processing on the feature image to obtain an adaptive threshold map, performing image segmentation processing on the adaptive threshold map, and segmenting different characters belonging to the same text area into the same segmented area to obtain a segmented map;

Step S500, combine the region candidate map and the segmentation map to perform text detection, and obtain a text detection result.

For step S100, the text image to be detected is acquired, wherein the text image is an image containing the text to be detected. The image can be a picture taken by a photographic device such as a camera, or a picture obtained by dividing a video into frames.

Input the text image into the trained text detection network. It is understandable that a large number of text images to be trained are input to the untrained text detection network, the text detection network is trained, and the parameters of the text detection network are adjusted through the loss function until the text detection network converges, then the text detection network Training is complete.

Referring to Fig. 2, for step S200, the feature information of the text area is extracted from the text image through the text detection network, and the feature image is obtained according to the feature information, including but not limited to the following steps:

Step S210, a plurality of residual blocks of the backbone network of the text detection network generate a plurality of basic feature maps containing feature information of the text area according to the text image;

In step S220, the feature pyramid network of the text detection network generates a plurality of fusion feature maps of different scales according to a plurality of basic feature maps;

Step S230, performing a concatenation operation and an upsampling operation on a plurality of fused feature maps to obtain a feature image.

For step S210, the backbone network is ResNet-50, and the backbone network is composed of multiple residual blocks. Multiple residual blocks of ResNet-50 generate multiple basic feature maps containing feature information of the text area according to the text image. Specifically, C3, C4 and C5 can be selected as the basic feature maps, and the corresponding step sizes are 8, 16 and 32 respectively. .

Specifically, the Resnet-50 network contains 49 convolutional layers and a fully connected layer; its network structure can be divided into seven parts, the first part does not contain residual blocks, and mainly performs convolution, regularization, and activation functions on the input. , The calculation of the maximum pooling. The second, third, fourth, and fifth parts of the structure all include residual blocks, and there are structures that do not change the size of the residual block but only change the dimension of the residual block. In the Resnet-50 network, the residual block has three layers of convolutions. The input of the Resnet-50 network is 224×224×3. After the convolution calculation of the first five parts, the output is 7×7×2048. The pooling layer will convert it into a feature vector, and the fully connected layer will process the feature vector. Computes and outputs class probabilities.

For step S220, the feature pyramid network of the text detection network generates multiple fusion feature maps of different scales according to multiple basic feature maps.

Wherein, the fused feature map includes a first feature map, a second feature map, a third feature map, a fourth feature map and a fifth feature map, respectively denoted as P3, P4, P5, P6, and P7. The size of the first feature map, the second feature map, the third feature map, the fourth feature map and the fifth feature map decrease in turn; the first feature map, the second feature map, the third feature map, the fourth feature map and The number of channels of the fifth feature map is the same.

Specifically, the size of the first feature map is 1/8 of the input image, the size of the second feature map is 1/16 of the input image, the size of the third feature map is 1/32 of the input image, and the size of the fourth feature map is The size is 1/64 of the input image, the size of the fourth feature map is 1/128 of the input image, the number of channels of the first feature map, the second feature map, the third feature map, the fourth feature map and the fifth feature map Unity is set to 256.

The feature pyramid network passes down the high-level features and supplements the low-level semantics, so that high-resolution and strong semantic features can be obtained, which is conducive to the detection of small targets. The feature pyramid network includes bottom-up, top-down, and horizontal connections; the two-layer features of the horizontal connection should be the same in spatial size, mainly to use the underlying positioning details, which is due to the feature map extracted by the bottom pyramid layer Contains more localization details, while the feature map extracted by the top pyramid layer contains more target feature information. It can use the feature maps extracted from different levels of pyramid layers for prediction, and extract features of different sizes from different layers of the network for prediction without adding additional calculations. The upper part only takes the bottom layer of the feature pyramid for prediction, and the second half makes separate predictions for all layers of the feature pyramid, and finally integrates all the prediction results.

For step S230, perform concatenation and upsampling operations on multiple fused feature maps to obtain feature images, specifically, perform upsampling on the first feature map, the result of the upsampling operation on the second feature map, and upsampling on the third feature map The result of the operation, the result of the upsampling operation on the fourth feature map, and the result of the upsampling operation on the fifth feature map are connected to obtain a feature image, and the feature image is fused with a multi-scale fusion feature map. The formula can be expressed as: F=P ₃ *Up(P ₄ )*Up(P ₅ )*Up(P ₆ )*Up(P ₇ ), F is a feature image, and Up is an upsampling operation.

With reference to Fig. 3, for step S300, feature image is carried out boundary detection processing to obtain the boundary of text area, the boundary of text area is positioned by a plurality of corner points to generate the region candidate box of text area, including but not limited to the following steps:

Step S310, performing boundary detection processing on the feature image, and representing the boundary of the character area in the feature image through the initial corner point;

Step S320, calculating the offset of different corner points, predicting the curvature according to the offset, and obtaining the curvature of the text area;

Step S330, increase or decrease the corner points according to the degree of curvature to locate the boundary representing the text region in the feature image, and then locate the boundary of the text region through multiple corner points, generate a region candidate frame for the text region, and obtain a region candidate map.

Previous corner detection methods usually use a polygon represented by a fixed number of points to describe the text area, for example, for horizontal text, use 2 points (upper left and lower right) to represent the text area; for multi-directional text, use a bounding box 4 points in (upper left corner, upper right corner, lower right corner, lower left corner) to represent these regions; but for scene characters with complex shapes and different scales, it is difficult to describe these text regions with a fixed number of points.

In this embodiment, the adaptive corner detection method first performs boundary detection processing on the feature image to obtain the boundary of the text area in the feature image, and the initial corner point represents the boundary of the text area in the feature image, such as the initial The corner points can be the four corner points on the boundary of the text area. The four initial corner points include the upper left corner, the upper right corner, the lower right corner, and the lower left corner. The coordinates of the four initial corner points are represented by clockwise two-dimensional coordinates , respectively (x1, y1), (x2, y2), (x3, y3), (x4, y4). The offset between different corner points is calculated according to the coordinates of the four initial corner points, and the curvature is predicted according to the offset to obtain the curvature of the text area. According to the curvature of the text area, adaptively add a new group or multiple pairs of corner point positioning to represent the boundary of the text area in the feature image, and then use multiple corner points to position the boundary of the text area to generate a text area The region candidate box, get the region candidate map. In addition, when the predicted text area is horizontal, only two corner points (upper left corner and lower right corner) on the text boundary can be reserved, which can reduce the amount of calculation. The number of corner points corresponds to the scale of the text area.

Referring to Fig. 4, for step S400, adaptive thresholding is performed on the feature image to obtain an adaptive threshold map, image segmentation is performed on the adaptive threshold map, and different characters belonging to the same text area are segmented into the same segmented area to obtain a segmented diagram, including but not limited to the following steps:

Step S410, calculating pixel values for the feature image, performing adaptive thresholding processing according to the pixel values, to obtain an adaptive threshold map;

Step S420, calculating the feature distance between characters according to the pixel value, and segmenting different characters belonging to the same character area into the same segmented area according to the feature distance, so as to perform image segmentation processing on the adaptive threshold map to obtain a segmented map.

Conventional image segmentation methods use the generated segmentation map to represent the probability that each pixel belongs to the text region. However, these text regions in segmentation maps cannot be separated from each other due to the overlap of text regions and the inaccurate prediction of text region pixels. In order to obtain text bounding boxes from segmentation maps, complex post-processing is required.

In this embodiment, pixel values are calculated for the feature image, adaptive thresholding processing is performed according to the pixel values, and pixel-level prediction is performed through the adaptive threshold to obtain an adaptive threshold map. The thresholds at different positions of the text area of the adaptive threshold map are different, and the threshold at the border of the text area is smaller than the threshold at the center of the text area. The feature distance between characters is calculated according to the pixel value, and different characters belonging to the same character area are segmented into the same segmented area according to the feature distance, so as to perform image segmentation processing on the adaptive threshold map to obtain a segmented map.

For step 500, the corner point candidate frame and the segmentation map are jointly optimized to finally obtain a visual text detection result; the text detection result includes the position of the text in the image and the text expression.

The self-adaptive text detection method, device, equipment and medium of the present application, combined with two ideas based on regression and segmentation methods, proposes an adaptive corner detection method and an adaptive threshold segmentation method, which generates different numbers of corners for text of different scales. Points are positioned, and the text segmentation map is generated by using the adaptive threshold method, and the generated corner candidate frames are combined for joint optimization to obtain visual text detection results; it can recognize and detect image text in arbitrary shapes and dense areas, and improve text detection. robustness.

The embodiment of the present application also provides an adaptive text detection device.

Referring to FIG. 5 , the adaptive text detection device includes an input module 110 , a feature extraction module 120 , an adaptive corner detection module 130 , an adaptive threshold segmentation module 140 and an output module 150 .

Wherein, the input module 110 is used for obtaining the text image to be detected, and the text image is input to the text detection network; the feature extraction module 120 is used for extracting the feature information of the text area from the text image, and obtains according to the feature information Feature image; the adaptive corner detection module 130 is used to perform boundary detection processing on the feature image to obtain the boundary of the text area, and locate the boundary of the text area through a plurality of corner points to generate the text area The region candidate frame, the number of the corner points corresponds to the ratio of the text region; the adaptive threshold segmentation module 140 is used to perform adaptive thresholding processing on the feature image to obtain an adaptive threshold map, and the self Adapting the threshold map to perform image segmentation processing, segmenting different characters belonging to the same text area into the same segmented area to obtain a segmented map; the output module 150 is used to perform text detection in conjunction with the region candidate map and the segmented map to obtain a text detection result .

It can be understood that the content in the embodiment of the adaptive character detection method is applicable to the embodiment of the adaptive character detection device, and the functions implemented in the embodiment of the adaptive character detection device are the same as those in the embodiment of the adaptive character detection method , and the beneficial effect achieved is also the same as that achieved by the embodiment of the adaptive text detection method.

The self-adaptive text detection method of the present application can be realized by using the PyTorch deep learning framework, and is carried out on the ubuntu20.04 system.

The adaptive text detection method can use the following data sets. The ICDAR2015 data set is a natural scene text image in the ICDAR robust reading competition. It includes 1000 training images and 500 test images; the data set is a scene taken by Google Glass Image, containing a large number of blurry text regions with large variation in text scale, whose text instances are labeled at the word level. The Total-Text dataset consists of 1255 training images and 300 test images, which contain more than 3 different text orientations: horizontal, multi-directional, and curved, etc.; the text in these images is labeled at the word level. The CTW1500 dataset contains 1000 training images and 500 test images, which contain pictographs of different scales, including multi-directional text, curved text, and irregularly shaped text; text regions in this dataset are marked with boundary points at the sentence level .

The ResNet-50 network is used for detection, and on this basis, the feature pyramid network is added as a feature extraction network to construct a multi-scale fusion feature map. First, pre-train the batch size of 8*100K iterations on the SynthText dataset, and then fine-tune the model on the other three open datasets. All networks are trained with Adam optimizer, and the learning rate is set to 1*10 ^-4 .

Mainly use three performance evaluation indicators for evaluation: precision rate (P), recall rate (R) and comprehensive index (F). P represents the ratio of the number of correctly detected text regions (TP) to the total number of detection results (E); R represents the ratio of the number of correctly detected text regions (TP) to the total number of real text annotations in Ground-Truth (G) ; F is a comprehensive indicator for evaluating the performance of text detection methods, and the value is obtained by calculating P and R.

Table 1 Comparison table of detection performance evaluation between adaptive text detection method and other methods

It can be seen from Table 1 that the comprehensive index F of the adaptive text detection method in this embodiment is 86.5%, 85.3% and 84.2% on the ICDAR2015, Total-Text and CTW1500 data sets, which is respectively improved by 0.8% and 84.2% compared with the PSENet network. 4.4% and 2%, and compared with the DBNet network, the comprehensive index F on the Total-Text and CTW1500 datasets increased by 0.6% and 0.8% respectively.

The embodiment of the present application also provides an electronic device. Referring to FIG. 6 , the electronic device includes a memory 220, a processor 210, a program stored on the memory 220 and operable on the processor 210, and a data bus 230 for realizing connection and communication between the processor 210 and the memory 220, the program When executed by the processor 210, the above adaptive text detection method is realized.

The electronic device may be any intelligent terminal including a tablet computer, a vehicle-mounted computer, and the like.

In general, for the hardware structure of the electronic device, the processor 210 may adopt a general-purpose CPU (Central Processing Unit, central processing unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits. Realization is used to execute related programs to realize the technical solutions provided by the embodiments of the present application.

The memory 220 may be implemented in the form of a read-only memory (ReadOnlyMemory, ROM), a static storage device, a dynamic storage device, or a random access memory (RandomAccessMemory, RAM). The memory 220 can store operating systems and other application programs. When implementing the technical solutions provided by the embodiments of this specification through software or firmware, the relevant program codes are stored in the memory 220 and called by the processor 210 to execute the implementation of this application. Example adaptive text detection method.

The input/output interface is used to realize information input and output.

The communication interface is used to realize the communication and interaction between this device and other devices, and the communication can be realized through a wired method (such as USB, network cable, etc.), or through a wireless method (such as a mobile network, WI FI, Bluetooth, etc.).

Bus 230 transfers information between the various components of the device, such as processor 210, memory 220, input/output interfaces, and communication interfaces. The processor 210 , the memory 220 , the input/output interface and the communication interface are connected to each other within the device through the bus 230 .

An embodiment of the present application provides a computer-readable storage medium. The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to make a computer execute the above-mentioned adaptive text detection method.

Those skilled in the art can understand that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware and an appropriate combination thereof. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit . Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). As known to those of ordinary skill in the art, the term computer storage media includes both volatile and nonvolatile media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data. permanent, removable and non-removable media. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cartridges, tape, magnetic disk storage or other magnetic storage devices, or can Any other medium used to store desired information and which can be accessed by a computer. In addition, as is well known to those of ordinary skill in the art, communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media . In the above description of this specification, the description with reference to the terms "one embodiment/example", "another embodiment/example" or "certain embodiments/example" means that the description is described in conjunction with the embodiment or example. A specific feature, structure, material, or characteristic is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the described specific features, structures, materials or characteristics may be combined in any suitable manner in any one or more embodiments or examples.

Those of ordinary skill in the art can understand that all or some of the steps in the methods disclosed above, the functional modules/units in the system, and the device can be implemented as software, firmware, hardware, and an appropriate combination thereof.

The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

If the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or part of the contribution to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including multiple instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method in each embodiment of the present application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), magnetic disk or optical disc, etc. can store programs. medium.

In the several embodiments provided in this application, it should be understood that the disclosed devices and methods may be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the above units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or can be Integrate into another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

Although the embodiments of the present application have been shown and described, those skilled in the art can understand that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principle and spirit of the present application. The scope of the present application is defined by the Examples and their equivalents.

The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the embodiments. Those skilled in the art can make various equivalent deformations or replacements without violating the spirit of the present application. These equivalent Any modification or replacement is within the scope defined by this embodiment.

Claims (10)

1. An adaptive text detection method, comprising:

acquiring a character image to be detected, and inputting the character image to a character detection network;

extracting feature information of a character area from the character image through the character detection network, and obtaining a feature image according to the feature information;

carrying out boundary detection processing on the characteristic image to obtain the boundary of the character area, positioning the boundary of the character area through a plurality of corner points, generating a candidate area frame of the character area, and obtaining a candidate area image, wherein the number of the corner points corresponds to the shape of the character area;

carrying out self-adaptive thresholding on the characteristic image to obtain a self-adaptive threshold image, carrying out image segmentation on the self-adaptive threshold image, and segmenting different characters belonging to the same character region into the same segmentation region to obtain a segmentation image;

and combining the region candidate graph and the segmentation graph to perform character detection to obtain a character detection result.

2. The adaptive text detection method according to claim 1, wherein the extracting feature information of the text region from the text image, and obtaining the feature image according to the feature information comprises:

generating a plurality of basic feature mappings containing feature information of the character area according to the character image by a plurality of residual blocks of a backbone network of the character detection network;

generating a plurality of fusion feature maps with different scales by a feature pyramid network of the character detection network according to the plurality of basic feature maps;

and performing connection operation and up-sampling operation on the plurality of fusion feature maps to obtain a feature image.

3. The adaptive text detection method according to claim 2, wherein the fused feature map comprises a first feature map, a second feature map, a third feature map, a fourth feature map and a fifth feature map; the sizes of the first feature map, the second feature map, the third feature map, the fourth feature map and the fifth feature map are sequentially reduced; the first characteristic diagram, the second characteristic diagram, the third characteristic diagram, the fourth characteristic diagram and the fifth characteristic diagram have the same channel number.

4. The adaptive text detection method according to claim 3, wherein the performing a join operation and an upsample operation on the plurality of fused feature maps to obtain a feature image comprises:

and performing connection operation on the first feature diagram, the result of performing up-sampling operation on the second feature diagram, the result of performing up-sampling operation on the third feature diagram, the result of performing up-sampling operation on the fourth feature diagram and the result of performing up-sampling operation on the fifth feature diagram to obtain a feature image.

5. The method according to claim 1, wherein the performing a boundary detection process on the feature image to obtain a boundary of the text region, locating the boundary of the text region through a plurality of corner points, and generating a region candidate frame of the text region to obtain a region candidate map comprises:

carrying out boundary detection processing on the characteristic image, and representing the boundary of a character area in the characteristic image through an initial corner point;

calculating the offset of different corner points, and predicting the curvature according to the offset to obtain the curvature of the character area;

and increasing or decreasing corner points according to the curvature to position the boundary of the character area in the representation characteristic image, further positioning the boundary of the character area through a plurality of corner points, generating an area candidate frame of the character area, and obtaining an area candidate image.

6. The adaptive text detection method according to claim 1, wherein the adaptively thresholding the feature image to obtain an adaptive threshold map, and the image segmentation processing the adaptive threshold map to segment different texts belonging to the same text region into the same segmentation region to obtain a segmentation map comprises:

calculating a pixel value of the characteristic image, and performing self-adaptive thresholding processing according to the pixel value to obtain a self-adaptive threshold map;

and calculating the characteristic distance between the characters according to the pixel values, and dividing different characters belonging to the same character region into the same divided region according to the characteristic distance so as to perform image division processing on the self-adaptive threshold map to obtain a divided map.

7. The adaptive text detection method according to claim 6, wherein the thresholds of different positions of the text region of the adaptive threshold map are different, and the threshold of the boundary of the text region is smaller than the threshold of the center of the text region.

8. An adaptive text detection apparatus, comprising:

the input module is used for acquiring a character image to be detected and inputting the character image to a character detection network;

the characteristic extraction module is used for extracting characteristic information of the character area from the character image and obtaining a characteristic image according to the characteristic information;

the self-adaptive corner detection module is used for carrying out boundary detection processing on the characteristic image to obtain the boundary of the character area, positioning the boundary of the character area through a plurality of corners, generating a candidate area frame of the character area and obtaining a candidate area image, wherein the number of the corners corresponds to the proportion of the character area;

the self-adaptive threshold segmentation module is used for performing self-adaptive thresholding on the characteristic image to obtain a self-adaptive threshold image, performing image segmentation on the self-adaptive threshold image, and segmenting different characters belonging to the same character region into the same segmentation region to obtain a segmentation image;

and the output module is used for carrying out character detection by combining the region candidate graph and the segmentation graph to obtain a character detection result.

9. An electronic device, comprising: memory, processor and computer program stored on the memory and executable on the processor, characterized in that the processor implements the adaptive text detection method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing one or more programs, the one or more programs being executable by one or more processors to implement the adaptive text detection method according to any one of claims 1 to 7.

Images