CN106339719A - Image identification method and image identification device - Google Patents

Abstract

The present application discloses an image recognition method and device. Firstly, normalize the image area of each sample image that matches the specified color tone, so as to obtain each image in the first image set, and then use the first image set to train the image classification device to obtain a trained image classifier, input the image to be recognized into the trained image classifier, and obtain a recognition result for the image to be recognized. Since the images in the first image set are obtained by processing the sample images, the proportion of the image area matching the preset image category in each image in the first image set is relatively increased. When zooming, the information loss of the image area matching the preset image category in the image is reduced. It can be seen that the method provided by this application can effectively reduce the number of images in the first image set, and improve the image classification while reducing costs. The training efficiency of the device.

Description

A method and device for image recognition

technical field

The present application relates to the field of information technology, in particular to an image recognition method and device.

Background technique

With the development of the information society and the growth of online social activities, people are more inclined to use images that are not restricted by region and language instead of words as the main medium of conveying words and expressions when conducting online social activities, which makes the rapid increase of images on the Internet. . How to utilize the massive images in the network has become one of the hot spots that people pay attention to in recent years.

Since images are different from text information, their content cannot be directly retrieved and classified through keywords, so for how to use images, the first problem to be solved is the identification of image content, that is, image recognition technology.

The existing image recognition technology mainly adopts the method of machine learning. Specifically, it is first necessary to manually classify images to determine image sets composed of images with different contents (such as image sets composed of landscape images and images composed of face images). set, image sets composed of pornographic images, etc.), and then for each image set of content, extract the common features (often feature vectors) between the images contained in the image set, and finally obtain the image set through training Finally, according to the feature models corresponding to various image sets, image recognition is performed on the received image to be recognized, and the category of the image to be recognized is determined.

Compared with manually setting and extracting feature vectors for image recognition, the feature model obtained through machine learning and training avoids the influence of human subjective factors, and can be continuously optimized through training, so the accuracy of image recognition is higher.

However, for machine learning methods, if you want to have a higher accuracy of image recognition, you first need a large number of images for learning and training feature models corresponding to image sets with different contents. If the images used for learning and training are too large If there are too few images, the accuracy of the determined feature model will decrease, which will affect the robustness of image recognition. If too many images are used for training, it will increase the resources of the machine learning method and affect the efficiency of machine learning.

Secondly, since there is a unified requirement for the size of the image used for training when training the feature model (for example, the unified image size is resolution: 100×100), it is also necessary to adjust the size of the image used for training (including: zooming in) , reduction, stretching, etc.), as shown in Figure 1, which leads to the loss of the features contained in the image, thus affecting the accuracy of machine learning (that is, affecting the accuracy of the final feature model), so that in order to ensure that the machine Learning accuracy requires further augmentation of training images.

FIG. 1 is a schematic diagram of feature loss contained in an image caused by image scaling of a high-resolution image.

Among them, the image on the left side is the original size, and the right side is the image after the reduced image size. In order to reflect the loss of features in the image after the reduced image size, the image after the reduced image size is enlarged again to the original size of the image. Size. It can be seen that the vein texture has been blurred, and if the vein texture is the feature to be extracted, the features of the reduced image size have been lost.

It can be seen that due to the above problems, the existing image recognition technology requires a large number of images for training, resulting in high cost of image recognition.

Contents of the invention

An embodiment of the present application provides an image recognition method, which is used to solve the problem in the prior art that a large number of images are required for training when a machine learning method is used for image recognition, resulting in an increase in the cost of image recognition.

An embodiment of the present application provides an image recognition device, which is used to solve the problem in the prior art that when a machine learning method is used for image recognition, a large number of images are required for training, resulting in an increase in the cost of image recognition.

The embodiment of the application adopts the following technical solutions:

A method for image recognition, comprising:

Determine the image to be recognized;

Inputting the image to be recognized into a pre-trained image classifier to obtain a recognition result output by the image classifier for the image to be recognized, wherein the first image set used by the image classifier for training is an image obtained by normalizing the image regions in the sample image that match the specified hue;

The specified hue is determined according to the hue of the image of the preset image category.

An image recognition device, comprising:

Determine the module and determine the image to be recognized;

A recognition module, inputting the image to be recognized into a pre-trained image classification module to obtain a recognition result output by the image classifier for the image to be recognized, wherein the first image used by the image classification module for training A concentrated image, obtained by normalizing the image regions in the sample image that match the specified hue;

The specified hue is determined according to the hue of the image of the preset image category.

The above at least one technical solution adopted in the embodiment of the present application can achieve the following beneficial effects:

Firstly, normalize the image area matching the specified tone of each sample image, so as to obtain each image in the first image set, and then train the image classifier through the first image set to obtain the trained image classifier, when When performing image recognition on the image to be recognized, the image to be recognized is input into the trained image classifier, so as to obtain a recognition result output by the image classifier for the image to be recognized. Wherein, since the images in the first image set are obtained by processing the sample images, the proportion of the image area matching the specified tone in each image in the first image set is relatively increased, so that even if it is necessary to Scaling the image can also reduce the feature loss of the image area matching the specified tone in the image, and increase the accuracy of the training result of the image classifier. It can be seen that the method provided by this application can be used without affecting the training effect. In this way, the requirement for the number of images in the first image set is effectively reduced, and the training efficiency of the image classifier is improved while reducing the cost.

Description of drawings

The drawings described here are used to provide a further understanding of the application and constitute a part of the application. The schematic embodiments and descriptions of the application are used to explain the application and do not constitute an improper limitation to the application. In the attached picture:

Figure 1 is a schematic diagram of the feature loss contained in the image caused by image scaling on a high-resolution image;

Fig. 2 is the image recognition process provided by the embodiment of the present application;

Fig. 3 is the process of training the convolutional neural network model provided by the embodiment of the present application;

Fig. 4 is a schematic diagram of determining the image area matching the specified tone in the sample image provided by the embodiment of the present application as an intermediate image;

FIG. 5 is a schematic structural diagram of a convolutional neural network model to be trained provided by an embodiment of the present application;

FIG. 6 is a schematic structural diagram of an image recognition device provided by an embodiment of the present application.

detailed description

In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with specific embodiments of the present application and corresponding drawings. Apparently, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.

As mentioned above, because the method of machine learning requires a large number of images for training, and in the case of unifying the image size of the images used for training, the number of required images needs to be further increased, so the current The number of images required for training is greatly increased in existing techniques.

Furthermore, since the method of machine learning is used, the feature models corresponding to different types of images are actually trained, so in the prior art, it is necessary to classify the images used for training in advance, so that the images can be classified according to the pre-classified The image is adjusted to the feature model, and finally the feature model with the correct rate of image recognition meeting the requirements is obtained (that is, the process of training the feature model). Wherein, the pre-classification of the images used for training usually relies on manual work, that is, it is necessary to manually classify each image in the image set used for training according to the content of the images.

However, since the prior art requires many images for training, a large amount of manual work for image classification is required, which increases the operating cost.

Furthermore, since humans mainly rely on human subjective feelings when classifying images, for some images that can be classified into different categories at the same time, the classification results of different people may not be consistent. Classifying images of different categories to train the feature model may have a negative impact on the training effect. However, since the existing technology requires more images for training, it is necessary to manually increase the number of images that can be classified into different categories at the same time. Screening, exclusion of the set of images used for training, or the need to further increase the images used for training.

Based on the above content, the embodiment of the present application provides a technical solution for image recognition that can reduce training images without affecting the effect of feature model training. The technical solutions provided by various embodiments of the present application will be described in detail below in conjunction with the accompanying drawings.

Fig. 2 is the image recognition process provided by the embodiment of the present application, which specifically includes the following steps:

S101: Determine an image to be recognized.

In the prior art, since the machine learning method usually consumes more resources to train the feature model, the process of training the feature model is generally performed by the server, and the terminal or the server can perform image recognition according to the trained feature model In this application, the image recognition process performed by the server is taken as an example for illustration.

Therefore, in the embodiment of the present application, the server may determine the image to be recognized for subsequent image recognition. Wherein, the image to be recognized may be determined by the server from locally stored images, or may be an image received by the server. For example, when a user publishes an image through a terminal, the image needs to be transmitted to the server through the terminal. , and then the server publishes the image online, then the server can determine the image as the image to be recognized when receiving the image at this time, and perform the subsequent image recognition process.

Furthermore, due to the high degree of informatization in the current society, there are also many images generated at each moment, so the number of images received by the server at each moment is also very large, so in this application, the server can also receive Randomly extract the obtained image, and determine the randomly extracted image as the image to be recognized, and carry out the subsequent image recognition process. Moreover, since the server can store the image locally on the server or in the server's database corresponding to the stored data after receiving the image, the server can also store the stored and unrecognized images when the operating pressure is low. The image is determined as the image to be recognized, and the subsequent image recognition process is carried out, that is, the server determines the image to be recognized from the locally stored images. Through the above method, the various types of images received and stored by the server can be better utilized. Image resource.

Of course, the above-mentioned method for determining the image to be recognized is only an embodiment provided by this application. In actual use, the same methods for determining the image to be recognized as in the prior art can also be used, which does not limit the use of this application. For the methods provided in the embodiments, this application does not specifically limit how to determine the image to be recognized.

Furthermore, since the image recognition of the image to be recognized is mainly for the following two purposes: one is to carry out risk control, and the other is to utilize image resources, among which, due to the relative impact of risk control on network security larger, so it is relatively more important, so the embodiment of this application will be described by performing image recognition on the image to be recognized for risk control as an example. Specifically, this application will use image recognition to determine whether the image is pornographic Take it as an example for subsequent descriptions.

It should be noted that in this application, the image recognition process can also be performed by a terminal, which can be a device such as a mobile phone, a personal computer, or a tablet computer. When the image recognition process is performed through a server, the server can be a separate device, or a network of multiple devices, that is, a distributed server. For the convenience of description, the image recognition process performed by the server will be used as an example for illustration later.

For example, assume that server A receives an image sent by a certain terminal, and the server A determines that the image is an image to be recognized, wherein the size of the image to be recognized is: resolution 1000×1000.

S102: Input the image to be recognized into a pre-trained image classifier, and obtain a recognition result output by the image classifier for the image to be recognized.

In the embodiment of the present application, after the server determines the image to be recognized, it can input the image to be recognized into the trained image classifier, so that the server can determine the image for the image classifier according to the output of the image classifier The recognition result of the image to be recognized. Moreover, the images in the first image set used by the image classifier for training are obtained by performing normalization processing on the image regions of the sample images that match the specified tone.

Specifically, in this application, since the image classifier may include: a convolutional neural network model, when the image classifier is a convolutional neural network model, the server may input the image to be recognized into the trained volume The product neural network model is used to obtain the image recognition result of the image to be recognized.

Moreover, as mentioned above, there are many training images and high training costs in the prior art. However, in the image recognition process provided by this embodiment of the present application, the image classifier required for image recognition is used for There are relatively few images for training, and the correct rate of image recognition of the trained image classifier is not affected, so the problems existing in the prior art can be avoided. In the following, the present application takes the image classifier as a convolutional neural network model as an example for illustration.

Specifically, in this application, the training of the convolutional neural network model is mainly through the process as shown in FIG. 3 .

Fig. 3 is the process of training the convolutional neural network model provided by the embodiment of the present application, including:

S1021: Determine a second image set composed of sample images.

In this embodiment of the present application, the server may first determine an initial sample image used for training the convolutional neural network model, and use an image set composed of each sample image as the second image set. Wherein, taking the use of the convolutional neural network model for image recognition of pornographic images as an example, the second image set may be an image set composed of sample images of three types of image content, including: sample images of pornographic content, non-pornographic Sample images for people content and sample images for non-people content.

Wherein, in terms of whether the content of the character content image involves pornographic content, the character content images can be classified into: pornographic content images and non-pornographic character content images. That is to say, pornographic content images and non-pornographic character content images are images of character content from the perspective of image content. The only difference is whether the image content involves pornographic content, and this difference is exactly the training of the convolution to be trained When using a neural network model, it is hoped that the convolutional neural network model to be trained can be learned, and when the trained convolutional neural network model is obtained, the trained convolutional neural network model can determine whether the image content involves pornography Therefore, in this application, the sample images in the second image may include: sample images of pornographic content and sample images of non-pornographic character content.

Further, since if only the image of the character content is used to train the convolutional neural network model to be trained, the finally obtained convolutional neural network model that has been trained will only have a relatively strong effect on the image to be recognized of the character content High recognition accuracy rate, but the recognition accuracy rate of non-person content images to be recognized is unpredictable, so in this application, the sample images in the second image set may also include: non-person content sample images.

Further, in this application, the number of sample images in the second image set may also be limited so that the number of sample images in the second image set is not greater than a preset number, for example, the preset number is 3000, Then the server can determine 3000 sample images.

Of course, the specific number of sample images in the second image set can be determined according to needs. The embodiment of this application only provides a solution, which does not constitute a limitation to this application. It should also be noted that if the second image is selected If the number of samples in the collection is too large (for example, 10,000, 100,000), it is difficult to reduce the sample images for training by the method provided in the embodiment of the present application, and this application does not require a large number of sample images.

Continuing with the above example, assume that when the server A trains the convolutional neural network model for identifying pornographic images, it needs to first determine 3000 sample images, including: sample images of pornographic content, sample images of non-pornographic character content, and Sample image for non-people content.

S1022: Classify the second image set.

In the embodiment of the present application, the same as the prior art, the server needs to classify each sample image orientation in the second image set, and add a mark to each sample image according to the classification result of each sample image.

Specifically, since the second image set may contain sample images of three types of image content in the application, the server may classify each sample image in the second image set into three categories according to the content of each sample image, as desired The trained convolutional neural network model can identify pornographic images, so the sample images can be divided into three types: sample images of pornographic content, sample images of non-pornographic character content, and sample images of non-personal content. In addition, the server can also add different labels to each type of sample image according to the classification results of each sample image in the second image set, so that when the convolutional neural network model is subsequently trained, the convolution The neural network model can determine the content of the input image according to different identifiers, and perform corresponding operations (such as calculating error values, calculating accuracy rates, and reversely adjusting parameters, etc.).

Further, in the embodiment of the present application, since the number of sample images used for training is limited, different from the prior art, in this application, the number of sample images of the three image contents in the second image set Consistency needs to be ensured. If the number of sample images of the three image contents in the second image set is inconsistent, and there is a large difference in quantity, it may make the feature learning of the convolutional neural network model incomplete for a certain image content. As a result, the image recognition accuracy rate of the convolutional neural network model is reduced. For example, assuming that the second image set contains 3000 sample images, wherein the numbers of sample images of the three image contents are: 1500, 1200, and 300 respectively, then it can be seen that in terms of the 300 sample images, the 300 sample images The image belongs to the sample image of the same kind of image content. It is further assumed that the same kind of image content has a total of 4 features a, b, c, and d. Since the number of the 300 sample images is small, the sample image of the image content The probability of covering all a, b, c, and d features is small, that is to say, due to the small number of sample images, the features contained in the image content of the image may be missed, so the features of the content image The probability of incomplete learning is high, which can easily lead to a decrease in the accuracy of image recognition of the convolutional neural network model. Therefore, in this application, following the above example, the number of sample images of the three kinds of image content in the second image set can be respectively: 1000, 1000, 1000, so that when training the convolutional neural network model, the convolution The characteristics of the three kinds of image contents are fully learned by the neural network model.

Furthermore, when adding a logo to each sample image, the logo can be added to the file name of the sample image according to a unified rule, for example, add a 3-digit logo after the file name, and use the symbol "-" as the The delimiter of the original file name, or adding a 3-digit English letter logo before the file name, etc., how to add the logo is not specifically limited in this application.

Continuing to use the above example, assuming that in step S1021, the number of sample images of pornographic content, sample images of non-pornographic character content, and sample images of non-personal content in the second image set determined by server A is 1000, then the Server A may classify each sample image in the second image set according to the content of each sample image, and add different identifiers to the above three types of sample images according to the classification results, as shown in Table 1.

Sample image category Logo added by sample image Sample Image for Sexual Content 001 Sample image for non-pornographic content 002 Sample images for non-people content 003

Table 1

Wherein, the logo can be added to the file name of the sample image, for example, if the name of a sample image is: 92e8647ajw1exg20dc07hx6x.jpg, and the content of the sample image is pornographic content, then the file of the sample image after adding the mark becomes: 92e8647ajw1exg20dc07hx6x-001.jpg.

S1023: Perform normalization processing on the image region of the sample image that matches the specified tone, to obtain images in the first image set used for training.

Wherein, the specified hue is determined according to the hue of the image of the preset image category through the prior art or manual experience.

In the embodiment of the present application, after the server determines the second image set and classifies and adds labels to each sample image in the second image set, since the size of each sample image in the second image set does not meet the size requirements of the input image used for training, and cannot be used for training, so the server also needs to process each sample image in the second image set to obtain the image that meets the size requirements of the input image used for training. Each image in the first set of images used for training.

Specifically, because in the prior art, when the size of each sample image is unified, only the stretching and zooming processing is performed on each sample image, as shown in Figure 1, which may lead to the loss of features contained in each sample image, so in this application In this method, firstly, for each sample image in the second image set, the server can determine the HueSaturationValue (HSV) color model of the sample image, that is, determine the hue, saturation value of each pixel of the sample image degree and lightness. Wherein, since the image usually expresses the value of each pixel with red, green, and blue ternary colors through the red, green, and blue (RedGreenBlue, RGB) color model, so in order to obtain the HSV color model of the sample image, the server can Use the following formula to convert the sample image from the RGB color model to the HSV color model.

$\left\{\begin{array}{c}\mathrm{<span\; class="notranslate">\; V</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; R</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; G</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; B</span>}}{\mathrm{<span\; class="notranslate">\; 3</span>}}\\ \mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; -</span>\frac{\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; \&times;</span><span\; class="notranslate">\; \&lsqb;</span>\mathrm{<span\; class="notranslate">\; min</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; R</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; G</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; B</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&rsqb;</span>}{\mathrm{<span\; class="notranslate">\; R</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; G</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; B</span>}}\\ \mathrm{<span\; class="notranslate">\; h</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; arccos</span>}<span\; class="notranslate">\; \{</span>\frac{<span\; class="notranslate">\; \&lsqb;</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; R</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; G</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; R</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; B</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&rsqb;</span><span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 2</span>}}{\sqrt{{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; R</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; G</span>}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; R</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; B</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; G</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; B</span>}<span\; class="notranslate">\; )</span>}}<span\; class="notranslate">\; \}</span>\end{array}\right.$

Among them, min(R, G, B) indicates that for each pixel in the sample image, the minimum value among the three values of R, G, and B of the pixel is taken.

Secondly, according to the HSV color model of the sample image, determine an image area corresponding to the preset image category of the sample image with a tone matching, as an intermediate image. Wherein, when the preset image category is the pornographic content image category, the hue corresponding to the pornographic content image category can be set as H∈[0, 116] according to artificial experience, then, the hue corresponding to the pornographic content image category is determined The image area to be matched is to determine the image area corresponding to the pixels whose tone value of the sample image is in the range of 0-116, and use this image area as an intermediate image. Moreover, when determining the image area of the sample image, the coordinate values of each pixel point whose hue value is in the range of 0 to 116 can be determined first, and the maximum coordinate value of the x-axis and the maximum value of the x-axis of each pixel point can be determined The coordinate value, the maximum coordinate value of the x-axis, and the maximum coordinate value of the x-axis determine the intermediate image corresponding to the sample image. As shown in Figure 4.

FIG. 4 is a schematic diagram of determining an image region matching the specified tone in the sample image as an intermediate image provided by an embodiment of the present application.

It can be seen that in Figure 4, the largest rectangular frame is the image boundary of the sample image, the gray area is each pixel point with a tone value ranging from 0 to 116, and the smallest dashed rectangular frame is the determined intermediate image, where the boundary of the intermediate image , determined by the x-axis maximum coordinate value, the x-axis maximum coordinate value, the x-axis maximum coordinate value, and the x-axis maximum coordinate value of each pixel point whose hue value is in the range of 0 to 116.

It should be noted that, when the corresponding intermediate image is determined through the sample image, it can be considered that the server has taken a screenshot of the sample image according to the image area of the sample image that matches the specified tone, and obtained The intermediate image, that is, as shown in FIG. 4 , the server intercepts the area of the smallest dotted rectangular frame in the sample image as the intermediate image.

Finally, after determining the intermediate image corresponding to the sample image, assuming that the image size of the intermediate image does not meet the size requirements of the input image used for training, the server can also normalize each intermediate image processing, and use the normalized intermediate images as images in the first image set, that is, normalize the intermediate images to obtain images in the first image set for training. Wherein, the normalization processing includes: according to the preset image size, adopting the same method as the prior art to scale and stretch the intermediate image, so that the image size of the intermediate image conforms to the preset image size, for example, assuming the The preset image size is a resolution of 256×256, and the image size of the intermediate image is a resolution of 300×400, then the server can scale and stretch the intermediate image and standardize the image size of the intermediate image to the resolution 256×256.

It should be noted that, in this application, the image category of pornographic content, that is, the image category corresponding to the sample image of pornographic content, that is, the sample image marked as 001 is added to the image.

Since each image in the first image set is obtained from each sample image in the second image set, there is a one-to-one correspondence between each image in the first image set and each sample image in the second image set , so it can be seen that through the processing of this step S1023, the server first intercepts the image area matching the specified color tone corresponding to the preset image category (such as pornographic content image category) in each sample image, so that for the first image Each image in the set is enlarged by the proportion of the image area matching the specified hue in the image compared to the image area matching the specified hue in the sample image corresponding to the image one-to-one. As shown in Figure 4, it can be seen that compared with the largest rectangular frame, the proportion of the gray area in the smallest dotted rectangular frame is relatively large, so even if the server passes subsequent normalization processing, the intermediate image is stretched And when scaling, the loss of features contained in each image used for training can also be reduced. Thereby avoiding the disadvantages existing in the prior art, due to the loss of image features due to the need for normalization processing, and the need to increase the number of images used for training, even if the server uses a small number of images for training, it can also achieve better results. training effect.

Furthermore, since the server extracts the intermediate image from the sample image, it can be considered that the server has removed a large number of interfering image regions. As shown in Figure 4, when training, the input image is the smallest dotted line Rectangular frame, relative to the largest rectangular frame, the smallest dotted rectangular frame has removed a large amount of useless background (that is, the interference image area), that is, the part of the largest rectangular frame that is more than the smallest dashed rectangular frame, so that the When training, the influence of image regions that do not match the specified hue on the training effect can be reduced.

For example, assuming that the images of pornographic content used for training are all images with white background, if there is no operation for determining the intermediate image as described in this application, then during training, the result that the correlation between white background and pornographic content images may be very strong , and we all know that the background color of an image is not directly related to whether the image is a pornographic image, so that the correct rate of image recognition obtained by training the convolutional neural network model is reduced. However, when the server determines each intermediate image according to the image area matched by the specified tone, the area of the white background in the image used for training is reduced a lot, so that the white background is no longer a main feature, so that it will not It affects the training of the convolutional neural network model, so that the image recognition accuracy rate of the convolutional neural network model obtained through training is higher.

S1024: According to the first image set, train the convolutional neural network model to be trained to obtain the trained convolutional neural network model.

In the embodiment of the present application, after the first image set is determined, the server can train the convolutional neural network model to be trained according to the first image set to obtain the trained convolutional neural network model .

Specifically, the following methods are used to train the convolutional neural network model:

First, the server determines the initialization parameters corresponding to each layer in the convolutional neural network model to be trained as the initialization model of the convolutional neural network model. Usually, the initialization parameter is determined randomly, and of course, it can also be determined manually based on experience, which is not limited in the present application.

Secondly, the server executes the following steps in a loop until the error value output by the convolutional neural network model to be trained reaches the first threshold and the image recognition accuracy reaches the second threshold:

Each image in the first image set is sequentially input into the convolutional neural network model to be trained, so that the features of the input training image are propagated forward to the output layer through the convolutional neural network model to be trained, and the output layer is calculated. According to the error value and the correct rate of image recognition, the parameters corresponding to each layer in the convolutional neural network model to be trained are reversely adjusted from the output layer according to the error value.

Therefore, when the server determines that the error value of the calculation output reaches the first threshold and the image recognition accuracy reaches the second threshold, it is determined that the training of the convolutional neural network model to be trained is completed, and the convolutional neural network model that has been trained is obtained. neural network model.

Through the image recognition method as shown in Figure 2, since the images in the first image set used by the server to train the image classifier are obtained by performing normalization processing on the image area of the sample image that matches the specified tone, so Even if the images in the first image set used for training have been normalized (that is, have been scaled and stretched), the feature loss contained in each image in the first image set can be greatly reduced, so that even if used for training The number of images in the first image set is small, and the image classifier with higher image recognition accuracy can also be trained, and, due to the reduction of the number of images in the first image set used for training, the image recognition cost reduction.

In addition, in step S101, the application does not specifically limit the size of the image to be recognized, but for an image to be recognized with a smaller image size, the size of the image to be recognized is small (for example, resolution 5×5), so the size of the image to be recognized The information contained in the recognition image is too little to be used, so in this application, the server can also determine the image whose size is larger than the threshold value as the image to be recognized according to the image size of the image.

Further, in step S102, specifically in step S1024, the structure of the convolutional neural network model may be as shown in FIG. 5 .

FIG. 5 is a schematic structural diagram of a convolutional neural network model to be trained provided by an embodiment of the present application.

It should be noted that only one activation layer is shown in Figure 5, but in the actual application process, the data output by each convolution layer needs to be activated by the activation layer before entering the next layer through the activation layer , as shown in the structural form of the first convolutional layer, the activation layer, and the first pooling layer in Figure 5, the data input to the first convolutional layer (ie, the images in the first image set), are generated by the first After the output of the convolutional layer, it can also be input into the activation layer for activation and output from the activation layer, and then input into the first pooling layer. In the same way, the data output by the first to fourth convolutional dimensionality reduction layers and the first to sixth convolutional feature extraction layers can also be input into their corresponding activation layers and output by their respective activation layers, and then Input the subsequent layers, but in Figure 5, in order to simplify the structure of the convolutional neural network model to be trained, not all activation layers are displayed.

In addition, it can be seen in FIG. 5 that the input layer is used to sequentially input each image in the first image set to each layer in the convolutional neural network model to be trained, wherein each convolutional dimensionality reduction layer is used to input The data of this convolutional layer is subjected to parameter dimensionality reduction and output to the next layer. For example, if the input data is 128 feature images with a resolution of 32×32, if the input is directly convolved and the features are extracted, assuming that 32 3×3 convolution kernels are used to extract features, the required The configured parameters are 128×32×3×3, and if a convolutional dimensionality reduction layer is used, the parameter dimensionality reduction can be performed through 32 1×1 convolution kernels, and the convolutional dimensionality reduction layer requires parameters of 128×32×1×1, output 32 feature maps, and then use 32 3×3 convolution kernels to extract features, the parameters that need to be configured can be reduced to 32×32×3×3, as shown in the comparison table 2.

Convolutional layer structure Need configuration parameters feature extraction directly 128×32×3×3 Dimensionality reduction before feature extraction 128×32×1×1+32×32×3×3

Table 2

Then, it can be seen from the structural diagram of the convolutional neural network model to be trained as shown in Figure 5 that the convolutional layer used to perform parameter dimensionality reduction on the data input to the convolutional layer and the feature used to perform feature analysis on the data input to the convolutional layer The extracted convolutional layers are adjacent to each other, so that the overall required parameters in the convolutional neural network model to be trained are reduced, and the training efficiency can be improved.

In addition, the first to fourth pooling layers can be seen in Figure 5, which are used to reduce the information redundancy generated after the input data (such as feature maps) undergo convolution operations, so as to improve the convolutional neural network model algorithm to be trained operating efficiency and robustness.

Further, the loss layer is used to calculate the error value of the image recognition result output by the convolutional neural network model to be trained after all the first image set is input into the convolutional neural network model to be trained, and the accuracy layer uses After calculating the accuracy rate of the image recognition result output by the convolutional neural network model to be trained after all the first image set is input into the convolutional neural network model to be trained, both the loss layer and the accuracy layer need Calculate the error value and the correct rate according to the logo added in the input image. For example, when calculating the correct rate, it is necessary to compare the image recognition results of each input image with the input Compare the logos of the images, if they are consistent, it is correct, if they are inconsistent, it is wrong, and finally determine the proportion of correctly recognized images in all input images, which is the image recognition accuracy rate of the convolutional neural network model to be trained .

Furthermore, the loss layer reversely adjusts the parameters of each layer in the convolutional neural network model to be trained by using the gradient descent method consistent with the prior art combined with the learning rate (ie, step size) according to the error value.

Further, the convolutional neural network model to be trained can judge whether the error value calculated by the loss layer reaches the first threshold and whether the image recognition accuracy calculated by the accuracy layer reaches the second threshold, if both, Then it is determined that the convolutional neural network model to be trained has been trained, and the trained convolutional neural network model is obtained. If at least one item is negative, the parameters of each layer can be adjusted in reverse through the loss layer adjustment, and the Each image in the first image set is input into the convolutional neural network model to be trained until the error value calculated by the loss layer reaches the first threshold and the image recognition accuracy calculated by the accuracy layer reaches the second threshold. threshold, where the first threshold and the second threshold can be set according to requirements, which is not specifically limited in this application.

Further, as shown in Figure 5, the convolutional neural network model to be trained adjusts the learning rate by the following formula:

lr=base_lr×γ×(floor(iter/stepsize)), where the lr is the learning rate (ie, step size) for each backpropagation, base_lr is the initialized learning rate parameter, stepsize and γ are constants, iter is the number of iterations.

In addition, in this application, after the server obtains the image recognition result of the image to be recognized through step S102, in order to increase the number of images of the preset image category in the first image set, the server can also , when it is determined that the image recognition result of the image to be recognized is the preset image category, according to the image area of the image to be recognized that matches the specified tone, determine the intermediate image corresponding to the image to be recognized, and the corresponding image to the image to be recognized After performing the normalization processing on the intermediate image, the intermediate image corresponding to the image to be recognized after the normalization processing is added to the first image set.

Of course, after the image recognition result is determined, how to use the image recognition result is not specifically limited in this application, and the above is only an implementation manner, and does not constitute a limitation to this application.

Further, this application does not limit the parameter structure of the convolutional neural network model and each layer, such as the size of the convolution kernel in the convolutional layer, the number of channels in the convolutional layer, the number of channels in the pooling layer, and The pooling step size and the like do not limit the specific pooling method, that is, the parameter structure of each layer in the convolutional neural network model can be set as required, and the application does not limit this.

It should be noted that, as shown in FIG. 1 , each step of the method provided by the embodiment of the present application may be executed by the same device, or the method may be executed by different devices. For example, the execution subject of step S1021 and step S1022 may be device 1, and the execution subject of step S1023 may be device 2; for another example, the execution subject of step S1021 may be device 1, and the execution subject of step S1022 and step S1023 may be device 2 ;etc.

Based on the image recognition process shown in FIG. 2 , the embodiment of the present application also provides an image recognition device correspondingly, as shown in FIG. 6 .

Fig. 6 is a schematic structural diagram of an image recognition device provided by an embodiment of the present application, including:

Determining module 201, determining the image to be recognized;

The recognition module 202, inputs the image to be recognized into a pre-trained image classifier, and obtains a recognition result output by the image classifier for the image to be recognized, wherein the image classifier uses the first The images in the image set are obtained by normalizing the image regions in the sample image that match the specified hue;

The specified hue is determined according to the hue of the image of the preset image category.

The device also includes:

The image set determination module 203 determines a second image set composed of sample images, and for each sample image, according to the HSV color model of the sample image, determines an image area in the sample image that matches the specified tone as an intermediate image , performing normalization processing on all intermediate images, and using a set of all intermediate images after normalization processing as the first image set used for training the image classifier.

The image classifier includes: a convolutional neural network model.

The device also includes: a training module 204, training the convolutional neural network model: determining the initialization parameters corresponding to each layer of the convolutional neural network to be trained, and performing the following steps in a loop until the convolutional neural network to be trained Until the error value output by the neural network model reaches the first threshold and the correct rate of image recognition reaches the second threshold, the training of the convolutional neural network model is completed: each image in the first image set is sequentially input into the volume to be trained A product neural network model, so that the features of the input training image are propagated forward to the output layer through the convolutional neural network model to be trained, and the error value and the image recognition accuracy rate of the output are calculated, according to The error value reversely adjusts parameters corresponding to each layer in the convolutional neural network model to be trained from the output layer.

The convolutional neural network model includes at least one convolutional layer, which is used to perform parameter dimensionality reduction on the data input to the convolutional layer. For each convolutional layer, the convolutional layer adjacent to the current convolutional layer The layer is used to extract features from the data input to this convolutional layer.

The image set determination module 204, when the recognition module 202 determines that the image recognition result of the image to be recognized is the preset image category, performs an image region matching the specified color tone in the image to be recognized After normalization processing, add to the first image set, and use the first image set added with the image to be recognized to retrain the image classifier.

The preset image category is a pornographic image category, and the hue value range of the specified hue corresponding to the pornographic image category is 0 to 116.

Specifically, the above-mentioned image recognition apparatus as shown in FIG. 6 may be located in one device, or in a system composed of multiple devices.

Those skilled in the art should understand that the embodiments of the present invention may be provided as methods, systems, or computer program products. Accordingly, the present invention can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each procedure and/or block in the flowchart and/or block diagram, and a combination of procedures and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions may be provided to a general purpose computer, special purpose computer, embedded processor, or processor of other programmable data processing equipment to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing equipment produce a An apparatus for realizing the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the instructions The device realizes the function specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby The instructions provide steps for implementing the functions specified in the flow chart or blocks of the flowchart and/or the block or blocks of the block diagrams.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

Memory may include non-permanent storage in computer readable media, in the form of random access memory (RAM) and/or nonvolatile memory such as read only memory (ROM) or flash RAM. Memory is an example of computer readable media.

Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can be implemented by any method or technology for storage of information. Information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Flash memory or other memory technology, Compact Disc Read-Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, Magnetic tape cartridge, tape magnetic disk storage or other magnetic storage device or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media excludes transitory computer-readable media, such as modulated data signals and carrier waves.

It should also be noted that the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus comprising a set of elements includes not only those elements, but also includes Other elements not expressly listed, or elements inherent in the process, method, commodity, or apparatus are also included. Without further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising said element.

Those skilled in the art should understand that the embodiments of the present application may be provided as methods, systems or computer program products. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The above descriptions are only examples of the present application, and are not intended to limit the present application. For those skilled in the art, various modifications and changes may occur in this application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.

Claims (10)

1. An image recognition method, characterized in that, comprising: Determine the image to be recognized; Inputting the image to be recognized into a pre-trained image classifier to obtain a recognition result output by the image classifier for the image to be recognized, wherein the images in the first image set used by the image classifier for training , which is obtained by normalizing image regions in the sample image that match the specified hue; The specified hue is determined according to the hue of the image of the preset image category. 2. The method according to claim 1, wherein the first set of images is obtained by the following method: determining a second image set consisting of sample images; For each sample image in the second image set, according to the hue-saturation-brightness HSV color model of the sample image, determine an image region in the sample image that matches the specified hue; and determining an intermediate image corresponding to the sample image according to an image area in the sample image that matches the specified tone; Normalize all intermediate images; A collection of all intermediate images after normalization processing is used as the first image set. 3. The method of claim 1, wherein the image classifier comprises a convolutional neural network model. 4. The method according to claim 3, wherein the convolutional neural network model is trained in the following manner: Determine the initialization parameters of each layer corresponding to the convolutional neural network model to be trained; The following steps are cyclically executed until the error value output by the convolutional neural network model model to be trained reaches the first threshold and the image recognition accuracy reaches the second threshold, and the training of the convolutional neural network model is completed: Each image in the first image set is sequentially input into the convolutional neural network model to be trained, so that the features of the input images are propagated forward to the output layer through the convolutional neural network model to be trained, Calculate and output the error value and the image recognition accuracy rate, and reversely adjust the parameters corresponding to each layer in the convolutional neural network model to be trained from the output layer based on the initialization parameters according to the error value. 5. the method for claim 4, is characterized in that, comprises at least one convolutional layer in the described convolutional neural network model, is used to carry out parameter dimensionality reduction to the data of inputting described convolutional layer; For each convolutional layer, the convolutional layer adjacent to the current convolutional layer is used to extract features from the data input to this convolutional layer. 6. The method of claim 1, further comprising: When it is determined that the image recognition result of the image to be recognized is that the image to be recognized belongs to the preset image category, the image area in the image to be recognized that matches the specified tone is subjected to normalization processing, and then added to the said first set of images; and The image classifier is retrained using the first image set added with the image to be recognized. 7. The method according to claim 1, wherein the preset image category is a pornographic image category; The hue value range of the specified hue corresponding to the pornographic image category is 0 to 116. 8. An image recognition device, characterized in that it comprises: Determine the module and determine the image to be recognized; A recognition module, inputting the image to be recognized into a pre-trained image classifier to obtain a recognition result output by the image classifier for the image to be recognized, wherein the first image used by the image classifier for training A concentrated image, obtained by normalizing the image regions in the sample image that match the specified hue; The specified hue is determined according to the hue of the image of the preset image category. 9. The device of claim 1, further comprising: The image set determination module determines a second image set composed of sample images, and for each sample image, according to the hue-saturation-brightness HSV color model of the sample image, determines the image area in the sample image that matches the specified hue, and determining an intermediate image corresponding to the sample image according to the image area in the sample image that matches the specified tone, performing normalization processing on all intermediate images, and using a set of all normalized intermediate images as the image The first image set used by the classifier for training. 10. The apparatus according to claim 1, wherein the image classifier comprises: a convolutional neural network model.