WO2021232865A1 - Object recognition method and device, apparatus, and medium - Google Patents

Abstract

An object recognition method and device, an apparatus, and a medium. A first recognition system acquires a target image containing multiple objects, and determines multiple sub-images corresponding to the target image. Each sub-image may contain at least one object. The first recognition system may then send the multiple sub-images of the target image to a second recognition system by means of remote communication, such that the second recognition system can perform recognition on the objects in the multiple sub-images. Since the first recognition system sends to the second recognition system the multiple sub-images corresponding to the target image rather than the whole target image, even if the target image contains sensitive information, respective portions of the sensitive information corresponding to the multiple sub-images are not sensitive, such that complete sensitive information cannot be determined easily, thereby reducing the risk of leaking the sensitive information.

Description

Object recognition method, device, equipment and medium

This application requires a Chinese patent application filed with the China Intellectual Property Office with the application number of 202010420378.2 and the application titled "Border-cloud-combined text recognition method, device and equipment" on May 18, 2020, and on June 24, 2020 The priority of a Chinese patent application filed with the China Intellectual Property Office, the application number is 202010588784.X, and the application name is "an object recognition method, device, equipment and medium", the entire content of which is incorporated into this application by reference.

Technical field

This application relates to the field of computer vision, and in particular to an object recognition method, device, equipment and medium.

Background technique

With the development of deep learning technology, the recognition accuracy of object recognition technology is getting higher and higher, and it is widely used in various industries. Under normal circumstances, recognizing objects (such as text, objects, etc.) in an image will have higher requirements on the data processing capabilities of the device. For example, in some scenes that have higher requirements on recognition speed and/or recognition accuracy, The data processing capacity of the device cannot be too low. As a result, some companies use multiple recognition systems to perform object recognition, for example, through the recognition system in their own edge data center and the recognition system in the cloud data center of the public cloud service provider to perform object recognition.

However, when the image to be recognized contains sensitive information, in an application scenario where multiple recognition systems perform object recognition, the risk of leakage of sensitive information is high.

Summary of the invention

In view of this, the embodiments of the present application provide an object recognition method to reduce the risk of leakage of sensitive information when recognizing an object in an image. This application also provides corresponding devices, equipment, computer-readable storage media, and computer program products.

In the first aspect, an embodiment of the present application provides an object recognition method, which can be executed by a first recognition system. The first recognition system may obtain a target image including a plurality of objects, where the object in the target image may be, for example, a text or an object. The first recognition system may determine multiple sub-images corresponding to the target image, and each sub-image may include at least one object. Then, the first recognition system may send multiple sub-images of the target image to the second recognition system through remote communication, So that the second recognition system can recognize the objects in the multiple sub-images.

Since the first recognition system sends multiple sub-images corresponding to the target image to the second recognition system, rather than the entire target image, even if the target image contains sensitive information, the second recognition system recognizes based on the multiple sub-images The information is usually only the parts of the sensitive information, and the combination of the parts of the sensitive information cannot be known. This makes when the parts of the sensitive information identified by the second identification system leak, because there are multiple parts between the parts. It is difficult to determine the sensitive information in the target image from a variety of information combinations, which is equivalent to that the sensitive information is not actually leaked, which can reduce the occurrence of sensitive information in the target image during the object recognition process. Risk of leakage.

In addition, when the first identification system is deployed in an edge data center and the second identification system is deployed in a cloud data center, the device for identifying objects in the sub-image can be a cloud data center with higher data processing capabilities instead of an edge data center. This not only makes the recognition accuracy and recognition efficiency of the object higher, but also does not require users to deploy high-performance servers on the edge side, which reduces the hardware cost requirements for users.

In a possible implementation manner, the first recognition system may obtain multiple sub-images corresponding to the target image by cropping the target image. Specifically, the first recognition system may obtain the pixel positions of multiple objects on the target image, and crop the target image according to the pixel positions of each object on the target image, so as to obtain multiple sub-images corresponding to the target image. Among them, the first recognition system may use its own object detection algorithm to detect multiple objects in the target image, and further obtain the pixel position of each object on the target image; or, the first recognition system may also use the first recognition system to detect multiple objects in the target image. The second recognition system recognizes multiple objects in the target image, and receives the pixel position of each object on the target image returned by the second recognition system.

In a possible implementation, when the first recognition system recognizes multiple objects in the target image through the second recognition system, the first recognition system may perform transformation processing on the target image. The image content of the object is transformed into other image content. For example, when the object is text, the text in the target image can be replaced with other text. Then, the first recognition system can send the transformed image obtained after transformation processing to the second recognition system, and the second recognition system uses a high-precision deep learning algorithm or object detection model to detect multiple transformed images in the transformed image The pixel position on the transformed image is returned to the first recognition system. In this way, the first recognition system can determine the pixel positions of the multiple objects on the target image according to the received pixel positions of the multiple transformed images. Since the second recognition system with a high-precision deep learning algorithm or object detection model detects the pixel position of the transformed object on the transformed image, the first recognition system can obtain a higher detection accuracy based on the pixel position returned by the second recognition system. The pixel position of the high object in the target image; and, in the process of obtaining the pixel position with higher accuracy by the first recognition system, the first recognition system sends the transformed image obtained through the transformation process to the second recognition system, and It is not the target image, which makes even the target image contains sensitive information, but the transformed image obtained based on the target image does not contain the sensitive information, so that the transformed image is transmitted to the second recognition system and in the second recognition system. In the process of object detection on the system, the risk of sensitive information in the target image being leaked is low.

In a possible implementation, the first recognition system may also receive a first recognition result for the plurality of sub-images returned by the second recognition system, and the first recognition result may be a result of the recognition results respectively corresponding to the multiple objects. gather. Then, the first recognition system may determine a second recognition result for the target image according to the first recognition result and the positional relationship of the objects in the multiple sub-images on the target image. The second recognition result may include the information of each object in the target image. The combination relationship, the combination relationship may be determined according to the position relationship of the object on the target image. For example, suppose that the first recognition result includes the recognition result "Zhang San" corresponding to object 1, the recognition result "XX city XX street XX community" corresponding to object 2, and the recognition result "Master" corresponding to object 3, so that according to the first According to the recognition result, the positional relationship between Object 1 and Object 2 in the target image, and the positional relationship between Object 1 and Object 3 in the target image, it is determined that the home address with the second recognition result of "Zhang San" is "XX City XX Street XX The educational background of "Community" and "Zhang San" is "Master" and so on.

In a possible implementation manner, the first recognition system may send multiple sub-images to the second recognition system based on a preset sequence. The preset sequence may be the sequence in which the multiple sub-images are cropped by the first recognition system, or it may be The cropping sequence is the order obtained after out-of-order adjustment, so that it is difficult for illegal users to obtain the target image based on the out-of-order adjusted sending order and multiple sub-image restoration, so that the sensitive information in the target image can be reduced by sending the sub-images out of order. Risk of leakage. Correspondingly, when the second recognition system returns the first recognition results for the multiple sub-images to the first recognition system, it may return the recognition results of the objects in each sub-image based on the preset order in order to facilitate the first recognition system According to the preset sequence, which recognition result corresponds to each sub-image is determined.

In a possible implementation manner, while returning the first recognition results of multiple sub-images, the second recognition system can also return the correspondence between each sub-image and its corresponding recognition result, so that the second recognition system is not required. The recognition result of the object in each sub-image is returned in a specific order, and the first recognition system can determine which recognition result each sub-image corresponds to according to the received correspondence.

In a possible implementation manner, the target image may include one image or multiple images. When the target image includes multiple images, taking the target image including the first image and the second image as an example, the multiple sub-images corresponding to the target image may include at least one sub-image corresponding to the first image and at least one corresponding to the second image Sub-image. Wherein, the first image and the second image may both contain sensitive information, or any one of them may contain sensitive information, or both of them may not contain sensitive information. In this way, when the first image and the second image contain sensitive information, the multiple sub-images corresponding to the target image sent by the first recognition system to the second recognition system can also be a mixture of sub-images of multiple images, which can increase The complexity of combining sub-images can increase the difficulty of combining multiple sub-images to obtain sensitive information, and further reduce the risk of sensitive information being leaked.

In a possible implementation, the first identification system can be deployed in an edge data center, and the second identification system can be deployed in a cloud data center, so that the higher data processing performance of the cloud data center can be used to obtain higher-precision object recognition As a result, the risk of sensitive information being leaked can be reduced, and users are not required to deploy costly high-performance servers in edge data centers.

In a possible implementation manner, the multiple objects included in the target image may include multiple characters, such as Chinese, English, numbers, symbols and other characters. Of course, it can also be other types of objects, such as trademarks, components and other objects.

In a possible implementation manner, the first recognition system receives the target image uploaded by the user and the sensitive indication information of the target image, so that the first recognition system can determine that the target image is an image containing sensitive information according to the sensitive indication information. In this way, the first recognition system can use any one of the possible implementations in the first aspect to complete object recognition on the target image containing sensitive information, and for the image that does not contain sensitive information, the first recognition system can send the image to The second recognition system performs object recognition. Exemplarily, the sensitive indication information may be, for example, a sensitive label added to the target image.

In the second aspect, an embodiment of the present application provides an object recognition method, which may be executed by a second recognition system. The second recognition system receives multiple sub-images corresponding to the target object from the first recognition system, where the target image may include multiple objects, and each sub-image of the target image may include at least one object. Then, the second recognition system may use a preset object recognition algorithm to recognize objects in each sub-image to obtain a first recognition result. The first recognition result may be a set of recognition results corresponding to multiple objects. Since the second recognition system performs object recognition on each sub-image of the target image, instead of directly performing object recognition on the entire target image, it is difficult for the second recognition system to know the combined relationship between multiple objects in the target image Therefore, even if the target image contains sensitive information and each part of the identified sensitive information leaks, it is difficult to determine the target image from a variety of information combinations due to the various possible information combinations between the various parts The sensitive information in the target image is equivalent to that the sensitive information has not actually leaked, which can reduce the risk of leakage of the sensitive information in the target image during the object recognition process.

In a possible implementation, the second recognition system sends the first recognition results for the multiple sub-images to the first recognition system, so that the first recognition system can further determine the target based on the first recognition system of the multiple sub-images. The second recognition result of the image.

In a possible implementation manner, the second recognition system further receives a transformed image from the first recognition system. The transformed image is an image obtained by performing transformation processing on the target image, wherein the object in the target image is undergoing transformation processing. The resulting transformation object can be different in image content. Then, the second recognition system can detect multiple transformed objects in the transformed image to obtain the pixel positions of multiple transformed images in the transformed image, such as using high-precision object detection algorithms or object detection models for detection, etc. The detected pixel position is returned to the first recognition system. Since the object in the target image has changed in the content of the transformed object obtained after the transformation process, the information presented by the transformed object is not part of the sensitive information, which makes the second recognition system unable to obtain information based on the transformed image The sensitive information contained in the target image can reduce the risk of sensitive information being leaked.

In a possible implementation, the second recognition system may return the recognition results of the objects in each sub-image to the first recognition system according to the order in which each sub-image of the target image is received, thereby completing the first recognition result. Feedback. In this way, the first recognition system can determine which recognition result corresponds to each sub-image of the target image according to the order in which the second recognition system sends the recognition results corresponding to the respective sub-images.

In a possible implementation, the second recognition system may use multiple processes to recognize objects in multiple sub-images in parallel. Compared with recognizing objects in multiple sub-images serially, the object recognition efficiency can be obtained. Significantly increased. Exemplarily, the multiple processes can be located on the same device or on different devices. For example, multiple sub-images of the target image can be divided into multiple parts, and the objects in the sub-images can be identified on different devices. While improving the efficiency of object recognition, it can also further increase the risk of sensitive information being leaked.

In a possible implementation manner, the target image may include one image or multiple images. When the target image includes multiple images, taking the target image including the first image and the second image as an example, the multiple sub-images corresponding to the target image may include at least one sub-image corresponding to the first image and at least one corresponding to the second image Sub-image. Wherein, the first image and the second image may both contain sensitive information, or any one of them may contain sensitive information, or both of them may not contain sensitive information. In this way, when the first image and the second image contain sensitive information, the multiple sub-images corresponding to the target image received by the second recognition system can also be a mixture of sub-images of multiple images, thereby increasing the number of sub-images. The complexity of the combination between multiple sub-images can increase the difficulty of combining multiple sub-images to obtain sensitive information, and further reduce the risk of sensitive information being leaked.

In a possible implementation, the first identification system can be deployed in an edge data center, and the second identification system can be deployed in a cloud data center, so that the higher data processing performance of the cloud data center can be used to obtain higher-precision object recognition As a result, the risk of sensitive information being leaked can be reduced, and users are not required to deploy costly high-performance servers in edge data centers.

In a possible implementation manner, the multiple objects included in the target image may include multiple characters, such as Chinese, English, numbers, symbols and other characters. Of course, it can also be other types of objects, such as trademarks, components and other objects.

In a third aspect, the present application provides an object recognition device, which can be applied to the first recognition system. The device includes: an acquisition module for acquiring a target image, the target image including a plurality of objects; and a determination module for Determine the multiple sub-images corresponding to the target image, each sub-image includes at least one object; a transmission module, configured to send the multiple sub-images to the second device, so that the second device can respond to the multiple sub-images Recognize objects in the middle.

In a possible implementation manner, the determining module is specifically configured to obtain the pixel positions of the multiple objects on the target image, and to determine the pixel positions of the objects on the target image. The target image is cropped to obtain multiple sub-images corresponding to the target image.

In a possible implementation manner, it is characterized in that the determining module is specifically configured to perform transformation processing on the target image to obtain a transformed image; the transmission module is further configured to send to the second recognition system The transformed image receives the pixel positions of multiple transformation objects in the transformed image returned by the second recognition system; the determining module is specifically configured to determine the pixel positions of the multiple transformation objects according to the pixel positions of the multiple transformation objects. The pixel positions of multiple objects on the target image.

In a possible implementation manner, the transmission module is further configured to receive the first recognition result for the multiple sub-images returned by the second recognition system; the determining module is further configured to According to the first recognition result and the positional relationship of the objects in the plurality of sub-images on the target image, a second recognition result for the target image is determined.

In a possible implementation manner, the transmission module is specifically configured to send the multiple sub-images to the second recognition system based on a preset sequence by the first recognition system, and receive the first recognition system. The second recognition system returns a first recognition result for the plurality of sub-images based on the preset order.

In a possible implementation manner, the target image includes at least a first image and a second image, and a plurality of sub-images corresponding to the target image includes at least a sub-image corresponding to the first image and the second image The corresponding sub-image.

In a possible implementation manner, the first identification system is deployed in an edge data center, and the second identification system is deployed in a cloud data center.

In a possible implementation manner, the multiple objects included in the target image include multiple characters.

In a possible implementation manner, the transmission module is further configured to receive the target image uploaded by the user and the sensitive indication information of the target image; the determination module is further configured to determine that the target image contains Images of sensitive information.

In a fourth aspect, the present application provides another object recognition device, which can be applied to a second recognition system. The device may include: a transmission module for receiving multiple sub-objects corresponding to the target object sent by the first recognition system through remote communication. An image, the target image includes a plurality of objects, and each sub-image includes at least one object; the recognition module is configured to perform object recognition on the plurality of sub-images to obtain a first recognition result for the plurality of sub-images, the The first recognition result includes the recognition result of the object in each sub-image.

In a possible implementation manner, the transmission module is further configured to send first recognition results for multiple sub-images to the first recognition system through remote communication.

In a possible implementation manner, the transmission module is further configured to receive a transformed image from the first recognition system, where the transformed image is an image obtained by transforming the target image; the device is also It includes: a detection module; the detection module is also used to detect multiple transformed objects in the transformed image to obtain the pixel positions of the multiple transformed objects in the transformed image; the transmission module is also used to Return the pixel positions of the plurality of transformed objects in the transformed image to the first recognition system.

In a possible implementation manner, the transmission module is specifically configured to sequentially return the recognition results of the objects in the respective sub-images to the first recognition system according to the receiving order of the respective sub-images of the target image .

In a possible implementation manner, the recognition module is specifically configured to use multiple processes to recognize images in the multiple sub-images in parallel.

In a possible implementation manner, the target image includes at least a first image and a second image, and a plurality of sub-images corresponding to the target image includes at least a sub-image corresponding to the first image and the second image The corresponding sub-image.

In a possible implementation manner, the first identification system is deployed in an edge data center, and the second identification system is deployed in a cloud data center.

In a possible implementation manner, the multiple objects included in the target image include multiple characters.

In a fifth aspect, the present application provides a computing device, which includes a processor, a memory, and a display. The processor and the memory communicate with each other. The processor is configured to execute instructions stored in the memory, so that the computing device executes the object recognition method in the first aspect or any implementation manner of the first aspect.

In a sixth aspect, the present application provides a computing device, which includes a processor, a memory, and a display. The processor and the memory communicate with each other. The processor is configured to execute instructions stored in the memory, so that the computing device executes the object recognition method in the second aspect or any implementation manner of the second aspect.

In a seventh aspect, the present application provides a computer-readable storage medium having instructions stored in the computer-readable storage medium, which when run on a computing device, cause the computing device to execute any of the above-mentioned first aspect or any of the first aspects. An object recognition method described in an implementation mode.

In an eighth aspect, the present application provides a computer-readable storage medium having instructions stored in the computer-readable storage medium, which when run on a computing device, cause the computing device to execute any of the above-mentioned second aspect or any of the second aspects. An object recognition method described in an implementation mode.

In a ninth aspect, this application provides a computer program product containing instructions, which when run on a computing device, causes the computing device to execute the object recognition method described in the first aspect or any one of the implementations of the first aspect .

In a tenth aspect, this application provides a computer program product containing instructions that, when run on a computing device, causes the computing device to execute the object recognition method described in the second aspect or any one of the implementations of the second aspect. .

On the basis of the implementation manners provided by the above aspects, this application can be further combined to provide more implementation manners.

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed in the description of the embodiments. Obviously, the drawings in the following description are only some of the implementations recorded in the present application. For example, for those of ordinary skill in the art, other drawings can be obtained based on these drawings.

Figure 1 is a schematic diagram of the architecture of an exemplary application scenario of this application;

2 is a schematic flowchart of an object recognition method in an embodiment of this application;

3 is a schematic diagram showing the coordinates of pixel positions in an embodiment of the application;

FIG. 4 is a schematic diagram of determining a second recognition result in an embodiment of this application;

FIG. 5 is a schematic diagram of interaction among users, edge data centers, and cloud data centers in an embodiment of the application;

6 is a schematic diagram of interaction among users, edge data centers, and cloud data centers in an embodiment of the application;

FIG. 7 is a schematic structural diagram of an object recognition device provided by an embodiment of the application;

FIG. 8 is a schematic structural diagram of yet another object recognition device provided by an embodiment of this application;

FIG. 9 is a schematic structural diagram of a computing device provided by an embodiment of this application;

FIG. 10 is a schematic structural diagram of another computing device provided by an embodiment of this application.

Detailed ways

The solutions in the embodiments of the present application will be described below in conjunction with the drawings in the present application.

The terms "first", "second", etc. in the description and claims of the application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific sequence or sequence. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, and this is merely a way of distinguishing objects with the same attributes in the description of the embodiments of the present application.

Object recognition technology usually refers to the use of computers and other equipment to process, analyze and understand images to identify various objects in the image. Among them, the objects in the image may be texts, objects, etc., for example. Taking recognizing text in an image as an example, it may be recognizing text in an image provided by a user (through a user terminal) in the application scenario shown in FIG. 1. Of course, the application scenario applicable to this application is not limited to the scenario example shown in FIG. 1, and may also be applied to other possible application scenarios.

As shown in Figure 1, users (which can be through user terminals and other devices) can provide images containing text to be recognized to an edge data center. The edge data center can be a collection of devices such as servers deployed on the edge. Provide users with corresponding business services on the edge, such as providing users with text recognition services. The edge data center represents the physical environment close to the user's device. Correspondingly, there is also a cloud data center, which can be a collection of equipment such as servers deployed on the cloud side. The cloud data center is usually farther from the user’s equipment than the edge data center. The cloud data center is dedicated to multiple regions. Of users provide cloud services. Generally, cloud data centers can have higher data processing performance (such as fast processing speed, high accuracy, etc.) than edge data centers, and can be used to provide better services than edge data centers For example, when the business service capabilities of the edge data center cannot meet the user's requirements, it can be handed over to the cloud data center for processing to meet the user's requirements. Or, the edge data center and the cloud data center can cooperate with each other to provide users with corresponding business services.

Because the edge data center's ability to recognize the text in the image is usually difficult to meet the user's requirements for the recognition accuracy and recognition speed of the object in the image, the edge data center can forward the image provided by the user to the cloud data through remote communication. Center, so that the cloud data center recognizes the text in the image based on the corresponding deep learning algorithm, and then the cloud data center transmits the recognition result to the user through the edge data center. For example, in a text translation scenario, users can send an image containing a piece of English text to the cloud data center through the edge data center; the cloud data center can identify the English content in the image, and pass the identified English content through the edge data The center delivers it to the user, who then uses the corresponding translation tool to translate the received English content into Chinese text.

However, when the image provided by the user contains sensitive information, such as the user’s work history, home address and other private information, the sensitive information in the image may be leaked on the cloud data center, for example, as identified by the cloud data center The result is stolen in the cloud, etc. Among them, sensitive information can refer to information that users are unwilling to leak, for example, it can be information that is improperly used or accessed or modified without authorization, which is not conducive to personal privacy rights enjoyed by individuals in accordance with the law.

In order to reduce the risk of sensitive information being leaked in the target image, some users choose to purchase and deploy high-performance servers in the edge data center, so that the edge data center has higher data processing capabilities, so that the image can be identified in the edge data center. The accuracy and efficiency of object recognition can usually meet the requirements of users. At the same time, it can also avoid the risk of sensitive information leakage caused by object recognition in the cloud data center. However, purchasing high-performance servers will dramatically increase user costs. However, if the edge data center with low data processing capability is used to identify objects in the image, the recognition speed is slow, the accuracy is low, and it is usually difficult to meet user requirements.

To this end, the embodiments of the present application provide an object recognition method, which can reduce the risk of leakage of sensitive information when recognizing objects in an image. Specifically, the edge data center may obtain a target image including multiple objects, and further determine multiple sub-images corresponding to the target image. For example, the edge data center may crop the target image into multiple sub-images, etc., where the determined Each sub-image of can include at least one object. Then, the edge data center can send multiple sub-images to the cloud data center through remote communication, and the cloud data center can identify the object in each sub-image, and obtain the recognition result corresponding to each sub-image. And send it to the edge data center, so that the edge data center obtains the recognition result for the target image based on the recognition result corresponding to each sub-image.

Since the edge data center sends multiple sub-images corresponding to the target image to the cloud data center, rather than the entire target image, even if the target image contains sensitive information, the cloud data center usually identifies information based on multiple sub-images. It is only the parts of sensitive information, and the combination of the parts of sensitive information cannot be known. This makes that when the parts of sensitive information identified by the cloud data center are leaked, there are multiple possible information between each part. Therefore, it is difficult to determine the sensitive information in the target image from a variety of information combinations, which is equivalent to that the sensitive information is not actually leaked, thereby reducing the risk of leakage of the sensitive information in the target image during the object recognition process.

At the same time, the device that recognizes the object in the sub-image is a cloud data center with higher data processing capabilities, rather than an edge data center. This not only makes the recognition accuracy and recognition efficiency of the object higher, but also does not require users to deploy on the edge. The high-performance server reduces the hardware cost requirements for users.

Next, various non-limiting specific implementations of object recognition will be described in detail.

Refer to FIG. 2, which is a schematic flowchart of an object recognition method in an embodiment of this application. This method can be applied to the application scenario shown in FIG. 1, of course, it can also be applied to other application scenarios, which is not limited in this embodiment. Exemplarily, the first identification system in the embodiment shown in FIG. 2 may be deployed in the edge data center (or deployed in the edge data center and user terminal) in the above-mentioned FIG. 1, for example, it may be a server system in the edge data center. , It can also be a software system that is deployed in the form of software on the equipment in the edge data center; the second identification system in the embodiment shown in Figure 2 can be deployed in the cloud data center shown in Figure 1, for example, It may be a server system in a cloud data center, or a software system, which is deployed in the form of software on devices in the cloud data center. The object recognition method shown in FIG. 2 may specifically include:

S201: The first recognition system acquires a target image, where the target image includes multiple objects.

The object recognition in this embodiment may refer to the process of recognizing the objects included in the target image based on the object recognition technology. Among them, the object to be recognized can be the text in the target image, such as Chinese, English, numbers, symbols, formulas, etc. Correspondingly, when the target image includes multiple objects, it can be that the target image contains multiple and multiple paragraphs. Or a variety of text content. Optionally, the object to be recognized may also refer to objects in the target image, such as trademarks, components, etc., of course, it may also be other types of objects.

In some scenarios, the information represented by the combination of multiple objects in the target image may be sensitive information. For example, the target image may contain information such as a person's name, home address, mobile phone number, and educational background, which are all personal privacy information, etc., as well as sensitive information in this embodiment.

In this embodiment, the target image containing sensitive information may be difficult to complete object recognition in the first recognition system. For example, the load of the first recognition system is high, and it is difficult to provide users with object recognition services for the target image; for example, the data processing performance of the first recognition system may be lower than that of the second recognition system. For object recognition, the accuracy and efficiency of possible recognition are low. Therefore, the second recognition system may perform object recognition on the target image. However, because the target image may contain sensitive information, if the first recognition system directly sends the target image to the second recognition system, the second recognition system will recognize the sensitive information on the target image, such as "Zhang San Sensitive information such as ""'s home address information, mobile phone number, and educational background may be leaked at the second identification system, thereby threatening "Zhang San"'s privacy.

For this reason, after acquiring the target image, the first recognition system may continue to perform step S202 and subsequent steps, so as to reduce the risk of leakage of sensitive information while completing object recognition. It should be noted that when the target image does not contain sensitive information, the technical solution of this embodiment may also be used for object recognition.

As an example, the first recognition system may determine whether the target image contains sensitive information based on the sensitive indication information. Specifically, the user can upload the target image to the first recognition system. At the same time, the user can also upload the sensitive indication information corresponding to the target image. The sensitive indication information may be, for example, a sensitive label added to the target image, which can be used to indicate Whether the target image contains sensitive information, the first recognition system can determine that the target object contains sensitive information according to the received sensitive indication information, and perform subsequent processing on the target image.

Optionally, the first recognition system can also provide users with object recognition services of two mechanisms, namely object recognition service 1 and object recognition service 2, where object recognition service 1 can be used to identify objects in images containing sensitive information. Objects, object recognition service 2 can be used to identify objects in images that do not contain sensitive information. In this way, the user can determine which object recognition service to choose on the first recognition system according to whether the image contains sensitive information. When the user selects object recognition service 1, the first recognition system can use the object recognition process described in this embodiment to complete the recognition of objects in the image, and when the user selects object recognition service 2, the first recognition system can directly The image is sent to the second recognition system to complete object recognition.

S202: The first recognition system may determine multiple sub-images corresponding to the target image, where each sub-image may include at least one object.

After obtaining the target image, the first recognition system may further determine multiple sub-images corresponding to the target image. Each sub-image can be a part of the target image, and each sub-image can include at least one object. Correspondingly, when the target image contains sensitive information composed of multiple objects, each component of the sensitive information can be located separately Different sub-images.

In a possible implementation, the first recognition system may first obtain the pixel positions of multiple objects on the target image on the target image, and compare the pixel positions of each object on the target image to the target image. By cropping, multiple sub-images can be obtained. Exemplarily, the pixel position of the object on the target image may be, for example, the coordinates of the center pixel point of the object on the target image, so that the first recognition system can determine a certain pixel range from the center pixel point according to the coordinates of the center pixel point The pixels within belong to the pixel points of the object on the target image, so that the pixel area of the image of the object on the target image can be obtained; or, the pixel position can also be the vertex of the image of the object on the target image Position, as shown in Figure 3, each row can represent the position of the rectangular frame of an object on the target image, which in turn represents the abscissa (32, 203, etc. in Figure 3) and the ordinate (such as 162, 124, etc. in Fig. 3), and the abscissa (165, 379, etc. in Fig. 3) and ordinate (182, 168, etc. in Fig. 3) in the lower right corner. Of course, the pixel position can also be expressed in other forms, which is not limited in this embodiment. Then, the first recognition system can crop the target image according to the pixel position of each object on the target image, so that multiple sub-images corresponding to the target image can be obtained.

As an example of determining the pixel position, the first recognition system may use an object detection algorithm configured on it to recognize the pixel position of each object on the target image. However, in some scenarios, the accuracy of the pixel positions detected by the first recognition system may be low. For example, when the first recognition system is deployed in an edge data center, due to the limitation of the data processing performance of the edge data center, the first recognition system It is difficult for the system to accurately and quickly detect the pixel position of the object in the target image, so when the first recognition system detects the object in the target image, there may be problems of low detection accuracy and slow detection speed.

Based on this, in another example of determining the pixel position, the second recognition system with higher data processing performance may also detect the pixel position corresponding to each object, so as to improve the detection accuracy of the pixel position. For example, the first recognition system can send the target image to the second recognition system; and the second recognition system can have high data processing performance and can support the operation of high-precision object detection algorithms. Therefore, the second recognition system can quickly The pixel positions of multiple objects on the target image are detected, and the detection accuracy of the pixel positions is high, and then the second recognition system can feed it back to the first recognition system, so that the first recognition system can obtain high detection accuracy The pixel position.

However, when the target image contains sensitive information, if the target image is sent to the second recognition system, the sensitive information on the target image may be leaked on the second recognition system, for example, when an illegal user is in the second recognition system. After the target image is stolen on the recognition system, the sensitive information on the target image will be known by illegal users, which will cause the leakage of sensitive information. Or, in the process of sending the target image by the first recognition system to the second recognition system (for example, in the communication link), leakage of sensitive information may easily occur.

Therefore, in yet another example of determining the pixel position, the first recognition system may first perform transformation processing on the target image to obtain the transformed image. Specifically, the first recognition system can use a preset object detection algorithm or object detection model to detect multiple objects on the target image, and transform the detected objects, such as object replacement, encryption, and masking. For any one or more of the processing, the object after the transformation processing can be called the transformation object, and the target image after the transformation processing can be called the transformation image, so that the sensitive information composed of multiple objects can be changed according to the object. It is transformed into non-sensitive information, which can realize the desensitization of information. In this way, even if an illegal user steals the target image (that is, the transformed image) after the transformation process, the target image after the transformation process can not be displayed. Contains the sensitive information, therefore, the original sensitive information on the target image may not be known by illegal users.

Then, the first recognition system can send the transformed image to the second recognition system. The second recognition system may use a preset deep learning algorithm to perform object detection on the received transformed image, determine the pixel position of each transformed object on the transformed image, and return the pixel position to the edge data center. It is worth noting that after the target image is transformed, there is usually a difference in content between the object on the target image and the transformed object on the transformed image, but the pixel position of the transformed object on the transformed image can be the same as the object on the target image. The pixel positions of are the same, or there is a certain correspondence between the two pixel positions. In this way, based on the pixel position of the transformed object on the transformed image, the pixel position of each object in the target image on the target image can be determined.

After the first recognition system determines the pixel position of the object in the target image on the target image, and crops the target image based on the pixel position to obtain multiple sub-images, it can continue to perform the subsequent step S203 and send the multiple sub-images to the second Recognition system to facilitate object recognition on the second recognition system.

It is worth noting that when the data processing performance of the second recognition system is higher than that of the first recognition system, the objects detected by the second recognition system based on the high-precision deep learning algorithm configured on it can be better. Comprehensive, that is, in addition to detecting the pixel position of the transformed object more accurately, the second recognition system can also detect the pixel position of the missing object on the transformed image. The missing object can be the first recognition system based on its object detection algorithm Objects that cannot be detected due to the limitation of accuracy, so that the first recognition system can correct multiple objects contained in the target image according to the pixel positions returned by the second recognition system, and can obtain relatively high-precision objects in each object. The pixel location on the target image.

S203: The first recognition system sends multiple sub-images corresponding to the target image to the second recognition system through remote communication.

In this embodiment, the first recognition system can perform remote communication with the second recognition system, such as communication based on HyperText Transfer Protocol (HTTP), etc., and multiple sub-images corresponding to the target image based on a preset sequence Send to the second identification system, as shown in Figure 2. Since what the first recognition system sends to the second recognition system is not the entire target image, but multiple sub-images of the target image, which makes the target image contain sensitive information and multiple sub-images of the target object are transmitted to the second recognition system The process is stolen by illegal users. Because illegal users cannot know the combination relationship between each sub-image, it is also difficult for illegal users to combine the information on each sub-image into the sensitive information included in the target image, which can reduce the sensitive information. Risk of leakage during transmission to the second identification system.

It is worth noting that the target image in this embodiment may be one image, and in some scenes, the target image may also include multiple images. Taking the target image at least including the first image and the second image as an example (the target image may also include three or more images), at this time, the multiple sub-images corresponding to the target image may include at least one corresponding to the first image The sub-image and at least one sub-image corresponding to the second image. The first image and the second image may both contain sensitive information, or one of the images may contain sensitive information while the other image does not contain sensitive information. In this way, the sub-image of the first image and the sub-image of the second image are mixed and sent to the second recognition system, which can further increase the difficulty of obtaining sensitive information on each image based on the combination of sub-images of different images, and reduce the leakage of sensitive information. risk. Of course, neither the first image nor the second image may contain sensitive information, which is not limited in this embodiment.

S204: The second recognition system recognizes an object in each sub-image, and obtains a first recognition result for multiple sub-images, and the first recognition result includes recognition results of multiple objects.

After receiving multiple sub-images, the second recognition system can use high-precision object recognition algorithms or object recognition models to recognize objects in each sub-image. For example, when the object is specifically a text, the second recognition system can use High-precision long-short-term memory (LSTM) and other algorithms perform text recognition, etc., so that the recognition result of the object in each sub-image can be obtained, so that the first recognition result of multiple sub-images can be obtained. The recognition result is the collection of the recognition results of the objects in each sub-image. In addition, the second recognition system can support high-precision object recognition algorithms or object recognition models (for example, the second recognition result can be deployed in a cloud data center with high data processing performance, etc.), so that the second recognition system performs object recognition on multiple sub-images. The first recognition result obtained by the recognition can usually achieve higher accuracy.

Exemplarily, the second recognition system may use multiple processes on it to recognize objects in multiple sub-images in parallel, for example, use process 1 to recognize objects in sub-image 1, use process 2 to recognize objects in sub-image 2, etc. Compared with the implementation of serially recognizing objects in each sub-image, the object recognition efficiency can be effectively improved.

As shown in FIG. 5 and FIG. 6, the first identification system can be deployed in one device or in multiple devices (device 1 and device 2 shown in FIG. 6). When the first identification system is deployed in one device, the device can complete the detection of the pixel position of the transformed object in the transformed image and the identification of the object in the target image; and when the first identification system is deployed in multiple devices In the middle, it may be that the device 1 completes the detection of the pixel position of the transformed object in the transformed image, and the device 2 completes the identification of the object in the target image, which is not limited in this embodiment.

S205: The second recognition system returns the first recognition result to the first recognition system through remote communication.

As shown in FIG. 2, the second recognition system may return the first recognition result obtained by the recognition to the first recognition system.

In a further possible implementation, to facilitate the first recognition system to determine which recognition result each sub-image of the target image corresponds to, the first recognition system may send multiple sub-images to the second recognition system based on a preset sequence, and the second recognition system The recognition system can record the receiving order of each sub-image when receiving multiple sub-images of the target image, and then return the recognition result corresponding to each sub-image in turn according to the receiving order of each sub-image. In this way, the first recognition system can determine that the first received recognition result is the recognition result corresponding to the first sent sub-image, and the second received recognition result is the recognition corresponding to the second sent sub-image The result, and so on. Of course, the first recognition system and the second recognition system can also negotiate other order correspondence rules. For example, the first recognition result received by the first recognition system is the recognition corresponding to the last sub-image sent by the second recognition system. As a result, the second recognition result received by the first recognition system is the recognition result corresponding to the second-to-last sub-image sent by the second recognition system, etc.

In addition to determining the corresponding relationship between the sub-images and the recognition results in the order of receiving and sending them, in other possible implementations, the first recognition system may also assign an image identifier to each sub-image and send it together with the sub-image. For the second recognition system, in this way, after the second recognition system determines the recognition result of the object in the sub-image, it can establish the corresponding relationship between the image identifier and the recognition result, so that the image identifier and the recognition result of each sub-image can be obtained. The corresponding relationship between the recognition results of the sub-images, so that the corresponding relationship and the first recognition result can be returned to the first recognition system together, so that the first recognition system determines which recognition each sub-image corresponds to according to the corresponding relationship As a result, there is no need to make requirements for the order in which the second recognition system sends the recognition results.

S206: The first recognition system determines a second recognition result for the target image according to the received first recognition result and the positional relationship of the objects in the multiple sub-images on the target image.

After receiving the first recognition result, the first recognition system can obtain the positional relationship of the objects in the multiple sub-images on the target image, and the positional relationship can reflect the combination relationship between different objects, and the first recognition system crops the target image It can be recorded locally, so that the first recognition system can combine the recognition results corresponding to each sub-image in the first recognition result according to the position relationship to obtain the second recognition result for the target image, as shown in Figure 2. , The second recognition result can reflect the information contained in the target image. For example, suppose that the first recognition result received by the first recognition system includes "Zhang San", "Li Si", "Master", "Doctor", "A Street", "C Street", and "B Community" , "D cell", "123XXXX8901", "123XXXX1098", the first recognition system determines the second recognition result according to the position relationship of the objects in the multiple sub-images on the target image as shown on the left side of FIG. 4 It is "Zhang San, with a master's degree in education, and his home address is 123XXXX8901 in Sub-District B, and the phone number is 123XXXX8901", and "Li Si, with a Ph.D degree in his education, his home address is in Sub-District C, Street D, and his phone number is 123XXXX1098", as shown on the right in Figure 4 Side shown.

As an example, the first recognition system may use a preset software development kit (SDK) to integrate the position of the object in each sub-image in the target image and the corresponding recognition result, and extract from the target image The structured data is extracted, such as the structured data shown on the right side of Figure 4, etc.

Because the second recognition result for the target image is obtained on the first recognition system, even if the target image contains sensitive information, the sensitive information is also stored in the first recognition system, and it is difficult to leak at the second recognition system . For example, when the first identification system is deployed in the edge data center on the customer edge side, and the second identification system is deployed in the cloud data center on the cloud side, the complete sensitive information can only be on the customer edge side, but not on the cloud side. , The risk of sensitive information being leaked is relatively low. Further, the customer edge side can return the extracted sensitive information (such as structured data) to the user, as shown in Figure 3 and Figure 4, the user can only perceive the interaction with the customer edge side, and there is no need to interact with the cloud side. Interaction, for the user, is equivalent to completing the recognition of the object in the target image at the edge of the customer, and not only the recognition accuracy is high, but the risk of the sensitive information obtained by the recognition is also low.

In addition, the first recognition system can use multiple processes on it to process the object recognition process of multiple images in parallel. For example, the first recognition system can use process 1 to perform the object recognition process of the target image, which may include the above-mentioned target image recognition process. The transformation processing, cropping, and structured data extraction, etc., use process 2 to perform the object recognition process of another image, including the transformation processing, cropping, and structured data extraction of the image using the above-mentioned similar implementation methods. The efficiency of recognizing objects in multiple images can be improved on the first recognition system.

For ease of understanding, the technical solutions of the embodiments of the present application will be introduced below in conjunction with an example of a scene where the object is specifically a text. Moreover, in this scenario, the first identification system can be deployed in an edge data center, and the second identification system can be deployed in a cloud data center. As shown in FIG. 5, this is a schematic flowchart of an object recognition method combined with a specific scene in an embodiment of this application, and the method may specifically include:

S501: The user uploads a target image to the edge data center, and the target image contains multiple paragraphs of text.

Wherein, the information obtained by combining multiple paragraphs of text in the target image may belong to the sensitive information described in the foregoing embodiment. For example, when multiple paragraphs of text in the target image include "Zhang San" (name), "XX community, XX street, XX city" (home address), "135XXXXXXXX" (mobile phone number), and "Master" (educational background). The combination of "Zhang San" and "XX Community, XX Street, XX City" can characterize the specific home address information of the person named "Zhang San"; the combination of "Zhang San" and "135XXXXXXXX" can characterize "Zhang San" Mobile phone number information; the combination of "Zhang San" and "Master" can characterize the academic information of "Zhang San", etc., and these information are all personal privacy information of "Zhang San" and also belong to the sensitive information in this embodiment. Of course, multiple objects in the target image can also be divided into a more fine-grained manner. For example, the above-mentioned home address information can be split into multiple objects. Then, the multiple objects in the above-mentioned target image can also include "Zhang San", "XX City", "XX Street", "XX Community", "135XXXXXXXX" (the phone number can also be split into multiple objects), and "Master", etc. The method for dividing objects in the target image is not limited in this embodiment.

Exemplarily, the user may upload a target image to the edge data center through a user terminal or a client. Further, when uploading a target image, the user can also upload sensitive indication information corresponding to the target image, so that the edge data center can determine that the target image contains sensitive information according to the sensitive indication information.

S502: The edge data center performs text replacement on multiple segments of text on the received target image to obtain pixel coordinates of the replacement text on the transformed image and the transformed image.

In specific implementation, the edge data center can use preset text detection algorithms, such as the MSER (Maximally Stable Extremal Regions) algorithm, etc., to detect multiple segments of text on the target image. Of course, the edge data center may also use other algorithms for text detection, which is not limited in this embodiment. Then, the edge data center can replace the detected text. Among them, replacement refers to replacing the content on the target image with other content, such as replacing the mobile phone number shown in FIG. 4 from "123XXXX8901" to "00000000000", etc. After the text on the target image is replaced, the new image obtained is the transformed image in step S502, and the pixel position of the replaced text in the transformed image and the pixel position of the text before the replacement in the target image can be maintained Unanimous. In this embodiment, the pixel position of the replaced text in the transformed image can be expressed as the pixel coordinates of the text in the transformed image.

It is worth noting that when the edge data center transforms the target image, it can not only replace the text in the target image, but also encrypt and mask the text in the target image. Among them, encryption refers to the use of corresponding encryption algorithms to encrypt the content on the target image. For example, the mobile phone number shown in Figure 4 can be encrypted from "123XXXX8901" to "234XXXX90123"; mask refers to using a mask Mask part of the object information on the target image and retain the remaining information, while the length of the object information remains unchanged. For example, a mask can be used to mask the mobile phone number "123XXXX8901" as shown in Figure 4, and the masked mobile phone number is "123* *****01". Of course, the transformation processing may also be other processing procedures, such as rearrangement (disrupting information in a certain order), etc., which is not limited in this embodiment.

Further, the edge data center can also randomly replace the text on the target image through a random algorithm. For example, it can calculate the corresponding random probability for each text. When the random probability of the text is greater than a preset value, the text can be The text is replaced, otherwise it will not be replaced, which can increase the difficulty of restoring the desensitized image. Of course, the text on the target image can also be replaced according to a certain rule, which is not limited in this embodiment. It is worth noting that the replaced text in this embodiment may refer to text that has not been replaced on the transformed image (for example, text with a random probability value less than a preset value) and replaced text.

S503: The edge data center sends the transformed image to the cloud data center through remote communication.

When the target image contains sensitive information, if the edge data center directly sends the target image to the cloud data center, the risk of leakage of the sensitive information during the transmission of the target image and on the cloud data center is high. Therefore, in this embodiment, the edge data center can transform the target image, specifically replacing the text on the target image, and send the transformed image obtained by the transformation to the cloud data center, so that the cloud data center can determine the transformation The pixel position of each paragraph of text on the image.

S504: The cloud data center can preprocess the transformed image, and use high-precision deep learning algorithms for text detection on the preprocessed transformed image, and obtain the multi-segment text on the transformed image on the preprocessed transformed image. Pixel position.

After the cloud center receives the transformed image, it can preprocess the transformed image. Exemplarily, the preprocessing may be, for example, performing processing such as cropping or perspective correction on the transformed image.

For example, in some scenes, there may be many blank areas in the transformed image (such as the background of the transformed image) that do not contain the transformed image, and the cloud data center does not need to perform object detection in the blank area. Therefore, the cloud data center After receiving the transformed image, the transformed image can be cropped to remove the blank area of the transformed image to obtain the cropped image; then, the cloud data center can use the preset deep learning algorithm to perform object detection on the cropped image , Determine the pixel position of the transformed object on the cropped image. The pixel position can also be the pixel position of the transformed object on the transformed image, or the transformation can be calculated based on the pixel position of the transformed object on the cropped image. The pixel position of the object on the transformed image. Among them, the cloud data center can use the high-precision psenet algorithm or retinanet algorithm to perform text detection on the transformed image, and determine the pixel position of each text on the transformed image. Of course, the algorithm for character recognition in the cloud data center is not limited to the above examples, and other applicable high-precision algorithms can also be used.

For another example, in some scenes, the replaced text on the transformed image is not a front view (such as not directly facing the object to take a photo, etc.). The cloud data center uses high-precision deep learning algorithms to detect the pixels of the transformed object. Before the position, perspective correction can be performed on the transformed image, for example, the transformed image can be rotated by a certain angle, and the characters in the front view after the rotation can be recognized. Correspondingly, the pixel position of the replaced text detected by the cloud data center using the deep learning algorithm is the pixel position of the replaced text on the transformed image after perspective correction processing.

S505: The cloud data center returns the pixel location and preprocessing information to the edge data center.

Because the cloud data center preprocesses the received transformed image, and the pixel position of the replaced text on the preprocessed transformed image may be different from the pixel position on the transformed image before the transformation process. Therefore, when the cloud data center returns the pixel position to the edge data center, it can also return preprocessing information, so that the edge data center can calculate the position of the replaced text on the transformed image before the preprocessing based on the preprocessing information. Pixel position. Of course, in other possible embodiments, the cloud data center can also calculate the pixel position of the replaced text on the transformed image before preprocessing based on the preprocessing information, and return the pixel position to the edge data center The embodiment shown in FIG. 5 is only used as an exemplary description, and is not used for limitation.

S506: The edge data center corrects the pixel positions of multiple replacement texts on the transformed image according to the received pixel positions and preprocessing information, and crops the target image according to the modified pixel positions to obtain a multi-sub image corresponding to the target image.

After the edge data center receives the pixel position returned by the cloud data center, it can further determine the pixel position of the multiple replacement text in the transformed image on the transformed image based on the received pixel position. For example, the edge data center may use the pixel position of the transformed object on the transformed image as the pixel position of the corresponding object in the target image on the target image; or the edge data center may calculate based on the received pixel position and preprocessing information Find out the pixel position of the multi-segment replacement text in the transformed image in the transformed image, and use the calculated pixel position to correct the pixel position of the multi-segment text detected by the text detection algorithm in the previous edge data center, and the corrected pixel position It can be used as the pixel position of each paragraph of text on the target image in the target image.

After the pixel position of the multiple replacement text in the transformed image is determined according to the received pixel position and preprocessing information, the pixel position is usually consistent with the pixel position of the text on the target image in the target image. Therefore, the edge data The center can crop the target image according to the pixel position of the replacement text in the transformed image to obtain multiple sub-images, and each sub-image can contain at least one paragraph of text in the target image.

For example, when the target image includes the words "Zhang San", "XX Street XX Community in XX City", "135XXXXXXXX" and "Master", by cropping the target image, at least 4 sub-images can be obtained, which can each contain The sub-images of "Zhang San", the sub-images containing "XX City XX Street XX Community", the sub-images containing "135XXXXXXXX" and the sub-images containing "Master", etc.; of course, the object in the target image is more detailed In the granularity division, at least 6 sub-images can also be determined, which can be the sub-images containing "Zhang San", the sub-images containing "XX City", the sub-images containing "XX Street", and the sub-images containing "XX Community". Sub-images, sub-images containing "135XXXXXXXX", and sub-images containing "Master"; of course, the sub-images obtained by cropping can be other numbers and contain other forms of text content, which is not limited in this embodiment.

S507: The edge data center sends the multiple sub-images obtained by cropping to the cloud data center.

In some possible implementations, the edge data center may adjust the sending order of each sub-image to the cloud data center. Specifically, the edge data center can cut out sub-images from the target image in a certain order according to the pixel positions of the text on the target image, such as cutting out the order of pixels on the target image from top to bottom (or from left to right) If the edge data center sends multiple sub-images to the cloud data center according to the order or reverse order of the cropped sub-images, illegal users may determine the combination of the sub-images according to the intercepted sub-image order, thereby increasing the target Risk of leakage of sensitive information on images. Therefore, the edge data center can adjust the sending order of each sub-image, so that illegal users based on the adjusted sending order of the sub-images, it is difficult to determine the combination relationship between the sub-images, and it is difficult to obtain the sensitive information in the target image. To a certain extent, it can reduce the risk of sensitive information being leaked. Of course, the edge data center may also send multiple sub-images to the second recognition system in the order of cropping the sub-images, which is not limited in this embodiment.

It is worth noting that the target image in this embodiment may be one image or may include multiple images. When the target image includes multiple images, the sub-images corresponding to the target image are a collection of sub-images respectively corresponding to the multiple images. In this way, mixing and sending sub-images of multiple images to the second recognition system can further increase the difficulty of combining sub-images based on different images to obtain sensitive information on each image, and reduce the risk of sensitive information being leaked.

S508: The cloud data center uses a high-precision deep learning algorithm to perform text recognition on each received sub-image, and obtain a recognition result of the text in each sub-image.

Exemplarily, the cloud data center may use a high-precision LSTM algorithm to recognize characters in each sub-image. Because the cloud data center can have high data processing capabilities and can support high-precision deep learning algorithms, the accuracy and efficiency of the text results obtained by recognizing the text in the sub-image in the cloud data center are usually high.

At the same time, although the target image may contain sensitive information, after the sensitive image is cropped into multiple sub-images, the information carried on each sub-image is usually part of the sensitive information, and multiple sub-images are sent to the cloud data center. Even when multiple sub-images are transmitted to the cloud data center or leaked in the cloud data center, it is difficult to obtain the target image based on the combination of the information on each sub-image because the combination relationship between the multiple sub-images cannot be known. In this way, the risk of leakage of sensitive information in the process of identification can be reduced.

S509: The cloud data center returns the recognition result of the text in each sub-image to the edge data center.

Exemplarily, the cloud data center may record the receiving order of each sub-image when receiving each sub-image, and send the recognition results of the characters in each sub-image to the edge data center in turn according to the recorded receiving order of each sub-image, thereby The edge data center can determine the recognition result corresponding to the text in which sub-image the recognition result is based on the order in which the recognition result is received.

Or, in other examples, when the cloud data center returns the recognition result, it can also return the corresponding relationship between the recognition result and the sub-image, so that the edge data center determines which recognition result corresponds to each sub-image according to the received correspondence Therefore, there is no need to require the cloud data center to send the corresponding sub-images in a certain order.

Further, while the cloud data center returns the first recognition result, it can also return the confidence level corresponding to the recognition result of the text in each sub-image to the edge data center. The confidence level can be used to indicate the reliability of each recognition result. Degree of trust. In this way, when the edge data center determines that the confidence of the recognition result of the text in the sub-image is lower than the preset value, it can choose to abandon the recognition result for the text in each sub-image and re-recognize the text in the sub-image, and When it is determined that the confidence of the recognition result of the text in the sub-image is higher than the preset value, the text recognition result for the target image may be further determined based on the recognition result of the text in each sub-image. Of course, the edge data center can also be based on user needs, regardless of whether the confidence of the recognition result of the text in the sub-image is higher than the preset value, the text recognition result for the target image is obtained according to the text recognition result of the sub-image, etc. This embodiment does not limit this.

S510: The edge data center extracts structured data from the target image according to the received recognition result of the object in each sub-image.

For example, the edge data center may use a preset SDK to integrate the position of the text in each sub-image in the target image and the corresponding recognition result, and extract structured data from the target image.

S511: The edge data center returns the extracted structured data to the user.

In the embodiment shown in FIG. 5, the detection process of the replacement text in the transformed image and the recognition process of the text in the sub-image are both performed by the same device on the cloud data center. In other possible implementation manners, it may also be performed by the same device on the cloud data center. Different devices in the cloud data center perform these two processes separately. As shown in Figure 6, the cloud data center can include equipment 1 and equipment 2, where equipment 1 can perform a process of detecting the pixel position of the replacement text in the transformed image, and equipment 2 can perform a process of recognizing text in multiple sub-images . Of course, the cloud data center can also include three or more devices, and all of them can participate in the above-mentioned text detection and text recognition process. For example, for multiple sub-images received by the cloud data center, multiple devices in the cloud data may simultaneously perform character recognition on different sub-images.

The object recognition method provided by the embodiment of the present application is described above with reference to FIGS. 1 to 6, and the object recognition apparatus provided by the embodiment of the present application and the computing device used to implement the function of the object recognition apparatus are described below with reference to the accompanying drawings.

As shown in FIG. 7, an embodiment of the present application also provides an object recognition device 700, which can be applied to the aforementioned first recognition system and execute the object recognition method executed by the aforementioned first recognition system. The embodiment of the present application does not limit the division of functional modules in the device 700. The following exemplarily provides a division of functional modules:

The obtaining module 701 is configured to obtain a target image, where the target image includes a plurality of objects;

The determining module 702 is configured to determine multiple sub-images corresponding to the target image, and each sub-image includes at least one object;

The transmission module 703 is configured to send the multiple sub-images to the second device, so that the second device can recognize objects in the multiple sub-images.

In a possible implementation manner, the determining module 702 is specifically configured to obtain the pixel positions of the multiple objects on the target image, and to determine the pixel positions of the objects on the target image. The target image is cropped to obtain multiple sub-images corresponding to the target image.

In a possible implementation manner, the determining module 702 is specifically configured to perform transformation processing on the target image to obtain a transformed image;

The transmission module 703 is further configured to send the transformed image to the second recognition system, and receive the pixel positions of multiple transformation objects on the transformed image returned by the second recognition system;

The determining module 702 is specifically configured to determine the pixel positions of the multiple objects on the target image according to the pixel positions of the multiple transformed objects.

In a possible implementation manner, the transmission module 703 is further configured to receive the first recognition result for the multiple sub-images returned by the second recognition system;

The determining module 703 is further configured to determine a second recognition result for the target image according to the first recognition result and the position relationship of the objects in the multiple sub-images on the target image.

In a possible implementation manner, the transmission module 703 is specifically used for the first recognition system to send the multiple sub-images to the second recognition system based on a preset sequence, and to receive the second recognition system The first recognition result for the multiple sub-images returned based on the preset order.

In a possible implementation manner, the target image includes at least a first image and a second image, and the multiple sub-images corresponding to the target image include at least a sub-image corresponding to the first image and a sub-image corresponding to the second image.

In a possible implementation manner, the first identification system is deployed in an edge data center, and the second identification system is deployed in a cloud data center.

In a possible implementation manner, the multiple objects included in the target image include multiple characters.

In a possible implementation manner, the transmission module 703 is further configured to receive the target image and the sensitive indication information of the target image uploaded by the user;

The determining module 702 is also used to determine that the target image is an image containing sensitive information.

According to the embodiment of the present application, the object recognition device 700 may correspond to the implementation of the object recognition method described in the embodiment of the present application, and the various modules and other operations and/or functions of the object recognition device 700 are designed to implement the first recognition system in FIG. 2 respectively. For the sake of brevity, the corresponding flow of each method executed will not be repeated here.

In addition, as shown in FIG. 8, an embodiment of the present application also provides an object recognition device 800, which can be applied to the aforementioned second recognition system and execute the object recognition method executed by the aforementioned second recognition system. The embodiment of the present application does not limit the division of functional modules in the device 800. The following exemplarily provides a division of functional modules:

The transmission module 801 is configured to receive multiple sub-images corresponding to the target object sent by the first recognition system through remote communication, where the target image includes multiple objects, and each sub-image includes at least one object;

The recognition module 802 is configured to perform object recognition on the multiple sub-images to obtain a first recognition result for the multiple sub-images, where the first recognition result includes the recognition result of the object in each sub-image.

In a possible implementation manner, the transmission module 801 is further configured to send first recognition results for multiple sub-images to the first recognition system through remote communication.

In a possible implementation manner, the transmission module 801 is further configured to receive a transformed image from the first recognition system, where the transformed image is an image obtained by performing transformation processing on the target image;

The device also includes: a detection module 803;

The detection module 803 is further configured to detect multiple transformation objects in the transformation image to obtain pixel positions of the multiple transformation objects in the transformation image;

The transmission module 801 is further configured to return the pixel positions of the multiple transformed objects in the transformed image to the first recognition system.

In a possible implementation manner, the transmission module 801 is specifically configured to sequentially return to the first recognition system the recognition of the objects in the respective sub-images according to the receiving order of the respective sub-images of the target image. result.

In a possible implementation manner, the recognition module 802 is specifically configured to use multiple processes to recognize images in the multiple sub-images in parallel.

In a possible implementation manner, the target image includes at least a first image and a second image, and the multiple sub-images corresponding to the target image include at least a sub-image corresponding to the first image and a sub-image corresponding to the second image.

In a possible implementation manner, the first identification system is deployed in an edge data center, and the second identification system is deployed in a cloud data center.

In a possible implementation manner, the multiple objects included in the target image include multiple characters.

According to the embodiment of the present application, the object recognition device 800 can correspond to the implementation of the object recognition method described in the embodiment of the present application, and the various modules and other operations and/or functions of the object recognition device 800 are designed to implement the second recognition system in FIG. 2 respectively. For the sake of brevity, the corresponding flow of each method executed will not be repeated here.

The object recognition apparatus 700 and the object recognition apparatus 800 described above can be implemented by computing devices, respectively. Figure 9 and Figure 10 respectively provide a computing device.

As shown in FIG. 9, the computing device 900 may be specifically used to implement the function of the object recognition apparatus 700 in the embodiment shown in FIG. 7.

The computing device 900 includes a bus 901, a processor 902, and a memory 903. The processor 902 and the memory 903 communicate through a bus 901.

The bus 901 may be a peripheral component interconnect standard (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into address bus, data bus, control bus and so on. For ease of representation, only one thick line is used in FIG. 9, but it does not mean that there is only one bus or one type of bus.

The processor 902 may be a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (digital signal processor, DSP), etc. Any one or more of the devices.

The memory 903 may include a volatile memory (volatile memory), such as a random access memory (random access memory, RAM). The memory 903 may also include non-volatile memory (non-volatile memory), such as read-only memory (ROM), flash memory, hard drive (HDD) or solid state drive (solid state drive). , SSD).

An executable program code is stored in the memory 903, and the processor 902 executes the executable program code to execute the object recognition method executed by the aforementioned first recognition system.

As shown in FIG. 10, the computing device 1000 may be specifically used to implement the function of the object recognition apparatus 800 in the embodiment shown in FIG. 8 above.

The computing device 1000 includes a bus 1001, a processor 1002, and a memory 1003. The processor 1002 and the memory 1003 communicate through a bus 1001.

The bus 1001 may be a PCI bus, an EISA bus, or the like. The bus can be divided into address bus, data bus, control bus and so on. For ease of representation, only one thick line is used to represent in FIG. 10, but it does not mean that there is only one bus or one type of bus.

The processor 1002 may be any one or more of processors such as CPU, GPU, MP, or DSP.

The memory 1003 may include a volatile memory (volatile memory), such as RAM. The memory 1003 may also include a non-volatile memory (non-volatile memory), such as ROM, flash memory, HDD or SSD.

The memory 1003 stores executable program codes, and the processor 1002 executes the executable program codes to execute the object recognition method executed by the aforementioned first recognition system.

The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium may be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The usable medium may be a magnetic medium, (for example, a floppy disk, a hard disk, and a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute the object recognition method performed by the above-mentioned first recognition system.

The embodiment of the present application also provides another computer-readable storage medium. The computer-readable storage medium may be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The usable medium may be a magnetic medium, (for example, a floppy disk, a hard disk, and a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute the object recognition method performed by the above-mentioned second recognition system.

The embodiment of the present application also provides a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on the computing device, the processes or functions described in the embodiments of the present application are generated in whole or in part.

The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website, a computer, or a data center through a cable (Such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) to another website site, computer or data center.

The computer program product may be a software installation package. In the case where any of the aforementioned object recognition methods needs to be used, the computer program product may be downloaded and executed on a computing device.

The description of the process or structure corresponding to each of the above drawings has its own focus. For parts of a process or structure that are not described in detail, please refer to related descriptions of other processes or structures.

Claims (20)

An object recognition method, characterized in that the method includes: The first recognition system acquires a target image, the target image including a plurality of objects; The first recognition system determines multiple sub-images corresponding to the target image, each sub-image includes at least one object; The first recognition system sends the plurality of sub-images to the second recognition system through remote communication, so that the second recognition system can recognize objects in the plurality of sub-images. The method according to claim 1, wherein the first recognition system to determine multiple sub-images corresponding to the target image comprises: Acquiring, by the first recognition system, the pixel positions of the multiple objects on the target image; The first recognition system crops the target image according to the pixel position of each object on the target image to obtain multiple sub-images corresponding to the target image. The method according to claim 2, wherein the acquiring, by the first recognition system, the pixel positions of the multiple objects on the target image comprises: The first recognition system performs transformation processing on the target image to obtain a transformed image; Sending the transformed image by the first recognition system to the second recognition system; Receiving, by the first recognition system, the pixel positions of the multiple conversion objects on the conversion image returned by the second recognition system; The first recognition system determines the pixel positions of the multiple objects on the target image according to the pixel positions of the multiple transformed objects. The method according to any one of claims 1 to 3, wherein the method further comprises: Receiving, by the first recognition system, a first recognition result for the plurality of sub-images returned by the second recognition system; The first recognition system determines a second recognition result for the target image according to the first recognition result and the positional relationship of the objects in the plurality of sub-images on the target image. The method according to any one of claims 1 to 4, characterized in that: The sending of the plurality of sub-images by the first recognition system to the second recognition system includes: Sending, by the first recognition system, the plurality of sub-images to the second recognition system based on a preset sequence; The receiving, by the first recognition system, the first recognition result for the multiple sub-images returned by the second recognition system includes: The first recognition system receives a first recognition result for the plurality of sub-images returned by the second recognition system based on the preset order. The method according to any one of claims 1 to 5, wherein the target image includes at least a first image and a second image, and the multiple sub-images corresponding to the target image at least include those corresponding to the first image. A sub-image and a sub-image corresponding to the second image. The method according to any one of claims 1 to 6, wherein the first identification system is deployed in an edge data center, and the second identification system is deployed in a cloud data center. The method according to any one of claims 1 to 7, wherein the multiple objects included in the target image include multiple texts. The method according to any one of claims 1-8, wherein before the first recognition system acquires the target image, the method further comprises: Receiving, by the first recognition system, the target image and the sensitive indication information of the target image uploaded by the user; The first recognition system determines that the target image is an image containing sensitive information. An object recognition device, characterized in that the device is applied to a first recognition system, and the device includes: An acquisition module for acquiring a target image, the target image including a plurality of objects; A determining module, configured to determine multiple sub-images corresponding to the target image, each of the sub-images includes at least one object; The transmission module is configured to send the multiple sub-images to the second device, so that the second device can recognize objects in the multiple sub-images. The device according to claim 10, wherein the determining module is specifically configured to obtain the pixel positions of the multiple objects on the target image, and according to the pixel positions of each object on the target image , Cutting the target image to obtain multiple sub-images corresponding to the target image. The device according to claim 11, wherein: The determining module is specifically configured to perform transformation processing on the target image to obtain a transformed image; The transmission module is further configured to send the transformed image to the second recognition system, and receive the pixel positions of multiple transformation objects on the transformed image returned by the second recognition system; The determining module is specifically configured to determine the pixel positions of the multiple objects on the target image according to the pixel positions of the multiple transformed objects. The device according to any one of claims 10 to 12, characterized in that: The transmission module is further configured to receive the first recognition result for the multiple sub-images returned by the second recognition system; The determining module is further configured to determine a second recognition result for the target image according to the first recognition result and the positional relationship of the objects in the multiple sub-images on the target image. The device according to any one of claims 10 to 13, characterized in that: The transmission module is specifically used for the first recognition system to send the multiple sub-images to the second recognition system based on a preset order, and to receive the second recognition system's return based on the preset order for all sub-images. The first recognition result of the multiple sub-images. The device according to any one of claims 10 to 14, wherein the target image includes at least a first image and a second image, and the plurality of sub-images corresponding to the target image includes at least the first image corresponding to the The sub-image of and the sub-image corresponding to the second image. The device according to any one of claims 10 to 15, wherein the first identification system is deployed in an edge data center, and the second identification system is deployed in a cloud data center. The device according to any one of claims 10 to 16, wherein the multiple objects included in the target image include multiple characters. The device according to any one of claims 10-17, wherein the transmission module is further configured to receive the target image and the sensitive indication information of the target image uploaded by the user; The determining module is also used to determine that the target image is an image containing sensitive information. A computing device, characterized by comprising a processor and a memory; The processor is configured to execute instructions stored in the memory, so that the computing device executes the method according to any one of claims 1-9. A computer-readable storage medium, characterized by comprising instructions, which when run on a computing device, causes the computing device to execute the method according to any one of claims 1 to 9.

Images