CN111445439B - Image analysis method, device, electronic equipment and medium - Google Patents

Abstract

The present application discloses an image analysis method, device, electronic device and medium. In the present application, at least one target image can be selected based on the face recognition result of at least one original image, the target image contains a human body area, semantic segmentation is performed on the target image, the background area in the target image is extracted, the first feature corresponding to the background area is extracted, and the first analysis result of the background area is calculated based on the first feature. By applying the technical solution of the present application, at least one target image containing a human body area can be selected from multiple original images, semantic segmentation can be performed on the target image and the first feature corresponding to the background area in the target image can be extracted, and the first feature can be analyzed to obtain a first analysis result, thereby improving the accuracy and practicality of image background evaluation in related technologies.

Description

Image analysis method, device, electronic equipment and medium

Technical Field

The present invention relates to image processing technology, and in particular to an image analysis method, device, electronic equipment and medium.

Background technique

With the development of the Internet, online education has become more and more popular. Online education and scientific research are flexible and can be learned at any time and place, which is convenient for learners to fully improve their skills. Compared with the traditional use of fixed classrooms, it is more mobile and convenient, and the pictures and audio are more visual and attractive. In the related art, the background image of the online classroom may be detected by random selection of video images plus manual screening, or by the full-image detection method of video frame by frame, and the background image of the online classroom may be evaluated and analyzed. However, the inventor found that when the background image of the online classroom is evaluated and analyzed by the above-mentioned technology, there are problems of inaccuracy in the evaluation of the background image and low practicality.

Summary of the invention

Embodiments of the present application provide an image analysis method, device, electronic device, and medium.

According to one aspect of the embodiments of the present application, an image analysis method is provided, characterized in that it includes:

Based on the face recognition result of at least one original image, selecting at least one target image, wherein the target image includes a human body area;

Performing semantic segmentation on the target image to extract a background area in the target image;

A first feature corresponding to the background area is extracted, and a first analysis result of the background area is calculated based on the first feature.

Optionally, in another embodiment based on the above method of the present application, the method further includes:

Identify at least one object in the target image, and determine a second analysis result of the object based on the identification result;

Based on the first analysis result and the second analysis result, a third analysis result of the target image is determined.

Optionally, in another embodiment of the method of the present application, the identifying at least one object in the target image and determining a second analysis result of the object based on the identification result includes:

Extracting at least one object region in the target image;

Extracting a second feature corresponding to the object region, and determining type information and/or attribute information of the object based on the second feature;

Based on the type information and/or the attribute information, the second analysis result is determined.

Optionally, in another embodiment of the method of the present application, the step of selecting at least one target image based on the face recognition result of at least one original image includes:

Using a preset face recognition model, detecting whether the original image contains a face image;

When it is determined that the original image contains a face image, obtaining a size ratio of the face image to the corresponding original image and a position of the face image in the original image;

Based on the size ratio and the location, the original image that meets the preset conditions is selected as the target image.

Optionally, in another embodiment of the method of the present application, the calculating a first analysis result of the background area based on the first feature includes:

Based on the first feature, determining the color type and color quantity corresponding to the background area;

Based on the color type and the color quantity, the first analysis result is determined.

Optionally, in another embodiment of the method of the present application, the calculating a first analysis result of the background area based on the first feature includes:

Calculating a matching degree between the first feature and a third feature corresponding to the human body region;

Based on the matching degree, the first analysis result is determined.

According to another aspect of an embodiment of the present application, a device for selecting an image is provided, comprising:

A selection module, configured to select at least one target image based on a face recognition result of at least one original image, wherein the target image includes a human body region;

An extraction module, used to perform semantic segmentation on the target image and extract a background area in the target image;

The calculation module is used to extract a first feature corresponding to the background area, and calculate a first analysis result of the background area based on the first feature.

According to another aspect of the embodiments of the present application, an electronic device is provided, including:

A memory for storing executable instructions; and

A display is used to display with the memory to execute the executable instructions to complete the operation of any of the above-mentioned image analysis methods.

According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided for storing computer-readable instructions, wherein the instructions, when executed, perform operations of any of the above-mentioned image analysis methods.

The beneficial effects brought by the technical solution provided by the embodiment of the present application include at least:

When the scheme of the embodiment of the present application is executed, face recognition is performed on the original image, and based on the face recognition result of at least one original image, at least one target image is selected, the target image contains a human body area, and semantic segmentation is performed on the target image, that is, the foreground area and the background area of the target image are separated, the background area in the target image is extracted, and then the first feature corresponding to the background area is extracted, and the first analysis result corresponding to the background area is calculated based on the first feature. Through the present application, at least one target image can be selected from multiple original images, the background area can be extracted from the target image, and then the first feature corresponding to the background area can be calculated to obtain the first analysis result, thereby improving the accuracy and practicality of evaluating the image background in the related art.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

FIG1 is a schematic diagram of the system architecture of the image analysis method of the present application;

FIG2 is a schematic diagram of a flow chart of an image analysis method proposed in this application;

FIG3 is a schematic diagram of a flow chart of an image analysis method proposed in this application;

FIG4 is a schematic diagram of a flow chart of an image analysis method proposed in this application;

FIG5 is a schematic diagram of the structure of the image analysis device of the present application;

FIG. 6 is a schematic diagram showing the structure of an electronic device according to the present application.

Detailed ways

Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. It should be noted that unless otherwise specifically stated, the relative arrangement of components and steps, numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present application.

At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual size ratio.

The following description of at least one exemplary embodiment is merely illustrative in nature and is not intended to limit the present application, its application, or uses.

Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

In addition, the technical solutions between the various embodiments of the present application can be combined with each other, but it must be based on the fact that ordinary technicians in the field can implement it. When the combination of technical solutions is mutually contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

It should be noted that all directional indications in the embodiments of the present application (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

The following describes the image analysis method according to the exemplary embodiment of the present application in conjunction with Figures 1 to 3. It should be noted that the following application scenarios are only shown to facilitate understanding of the spirit and principles of the present application, and the embodiments of the present application are not limited in this regard. On the contrary, the embodiments of the present application can be applied to any applicable scenario.

FIG. 1 is a schematic diagram showing an exemplary system architecture 100 to which an image analysis method or an image analysis device according to an embodiment of the present application can be applied.

As shown in Fig. 1, the system architecture 100 may include one or more of terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

It should be understood that the number of terminal devices, networks and servers in FIG1 is only illustrative. According to implementation requirements, there may be any number of terminal devices, networks and servers. For example, the server 105 may be a server cluster composed of multiple servers.

Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Terminal devices 101, 102, 103 can be various electronic devices with display screens, including but not limited to smart phones, tablet computers, portable computers, desktop computers, etc.

The terminal devices 101, 102, and 103 in the present application may be terminal devices that provide various services. For example, a user selects at least one target image based on the face recognition result of at least one original image through the terminal device 103 (which may also be the terminal device 101 or 102), and the target image includes a human body region; performs semantic segmentation on the target image, extracts the background region in the target image; extracts the first feature corresponding to the background region, and calculates the first analysis result of the background region based on the first feature.

It should be noted here that the image analysis method provided in the embodiment of the present application can be executed by one or more of the terminal devices 101, 102, 103, and/or the server 105. Accordingly, the image analysis device provided in the embodiment of the present application is generally arranged in the corresponding terminal device, and/or the server 105, but the present application is not limited to this.

The present application also proposes an image analysis method, device, target terminal and medium.

FIG2 schematically shows a flow chart of an image analysis method according to an embodiment of the present application. As shown in FIG2 , the method includes:

S201, selecting at least one target image based on a face recognition result of at least one original image, wherein the target image includes a human body region.

It should be noted that the present application does not specifically limit the device for acquiring the original image, for example, it can be a smart device or a server. The smart device can be a PC (Personal Computer), a smart phone, a tablet computer, an e-book reader, an MP3 (Moving Picture Experts Group Audio Layer III) image selection device, an MP4 (Moving Picture Experts Group Audio Layer IV) image selection device, a portable computer and other mobile terminal devices with display functions, etc.

Optionally, the original image is not specifically limited in the present application, that is, the original image in the present application can be any image information. In a preferred embodiment, the original image can be an image containing a human body region, etc.

In addition, the present application does not specifically limit the number of original images, for example, it can be one or ten.

Furthermore, after acquiring the original image, in order to ensure that the image quality of the target image finally selected meets the requirements, the present application may first detect the corresponding clarity parameters and brightness parameters for each original image.

It is understandable that the present application can calculate the clarity value of each image based on the pixel neighborhood parameters corresponding to each image. Among them, the pixel neighborhood parameters in the present application can be parameters that reflect the pixel neighborhood difference of the image. It is understandable that the higher the pixel neighborhood difference corresponding to an image, the higher the clarity it reflects. Therefore, the present application can generate corresponding thresholds based on the pixel neighborhood parameters to remove original images whose image clarity is lower than the preset threshold, and then use the remaining original images that meet the clarity standard as candidate images for determining the target image.

In addition, in order to ensure the clarity of the target image selected from the original image, the present application can detect the brightness parameter of each image to be selected, and then determine the image brightness of each original image through the brightness parameter.

Image brightness refers to the brightness of the image, and the unit is candela per square meter (cd/m2). Image brightness is a continuum of perception from white surface to black surface, which is determined by the reflectance coefficient. In addition, brightness refers to the degree of brightness of the light shining on the scene or image. When the image brightness increases, it will appear dazzling or glaring, and when the brightness is smaller, the image will appear gray.

Furthermore, after the above processing is performed on the original image, an original image to be processed is obtained, and the original image to be processed is identified using face recognition technology. According to the recognition result of the face recognition technology, at least one target image is selected from the original image to be processed, and the target image contains a human body area.

S202, performing semantic segmentation on the target image to extract the background area in the target image.

Among them, semantic segmentation refers to dividing all pixels in the image into corresponding categories to achieve pixel-level classification. As for extracting the background area of the target image, it can be achieved using character image segmentation technology. Among them, character image segmentation refers to separating the foreground and background of character photos. The goal is to classify each pixel of the input image into foreground and background, and obtain a classification map at the pixel level. Specifically, the extraction of the background area can be based on image segmentation algorithms, such as threshold segmentation algorithms, edge-based segmentation algorithms, region expansion algorithms, watershed algorithms, etc. These algorithms are only applicable when the image is relatively simple, and it is difficult to obtain ideal segmentation results for images with complex colors. Furthermore, for FCN and its derivative methods such as SegNet and DeepLab, the above technology aims to achieve general image semantic segmentation to achieve high-precision character image segmentation.

Furthermore, the present application can segment the original image using preset segmentation conditions, and then adjust the segmentation threshold according to the grayscale value and number of image pixels, so as to determine the background area in the target image, thereby realizing automatic extraction of the background area.

S203: extract a first feature corresponding to the background area, and calculate a first analysis result of the background area based on the first feature.

The first feature includes image parameters such as color parameters, clarity parameters, etc. The first analysis result is used to determine whether the original image corresponding to the background area meets the target image.

Taking the online education industry as an example, the first interaction between teachers and students often starts with the profile photo of one user. For example, a standard and clear teacher photo can often attract many students to make appointments for classes, but at this stage, there are often many problems with teachers uploading photos. Including image quality, image layout, image background, and so on. Furthermore, in terms of image quality, there are often problems with photos being too small, insufficient resolution, and irregular photo shapes; and in terms of image layout, there may be problems such as faces being too large or too small, the characters in the photos not being centered, and not being in the front face. As for the image background, there may be problems such as the background being too simple, the background of the picture being too messy, and the characters holding pets. In addition, in terms of the emotions of the characters in the image, there may also be problems such as the characters being too serious.

Furthermore, the existing audit solutions at this stage, namely simple machine audit, manual audit and combination of simple audit and manual audit, each have their own shortcomings. For example, for simple machine audit, simple image recognition dimensions such as image size, resolution, clarity and other indicators are used to conduct preliminary screening of images. Manual audit refers to the use of manual judgment and scoring of images. The combination of machine and manual audit means that the machine gives an approximate score, and the dimension detection of images with abnormal scores given by the machine is manually reviewed.

In addition, for the field of online education, it is essential to create a warm and comfortable teaching environment when teachers carry out teaching activities. However, it is difficult to quantify and evaluate whether the layout of the classroom is appropriate and comfortable. Therefore, how to comprehensively select images with good background and characters from a large number of original images has become a problem that technicians in this field need to solve.

It should be noted that after the background area corresponding to the original image is acquired, the first analysis result of the background area can be calculated based on the first features of each background area, such as color parameters and clarity parameters.

When the scheme of the embodiment of the present application is executed, face recognition is performed on the original image, and based on the face recognition result of at least one original image, at least one target image is selected, the target image contains a human body area, and semantic segmentation is performed on the target image, that is, the foreground area and the background area of the target image are separated, the background area in the target image is extracted, and then the first feature corresponding to the background area is extracted, and the first analysis result corresponding to the background area is calculated based on the first feature. Through the present application, at least one target image can be selected from multiple original images, the background area can be extracted from the target image, and then the first feature corresponding to the background area can be calculated to obtain the first analysis result, thereby improving the accuracy and practicality of evaluating the image background in the related art.

FIG3 schematically shows a flow chart of an image analysis method according to an embodiment of the present application. As shown in FIG3 , the method includes:

S301, using a preset face recognition model, detecting whether the original image contains a face image.

S302: When it is determined that the original image contains a face image, the size ratio of the face image to the corresponding original image and the position of the face image in the original image are obtained.

It is understandable that the present application can further limit whether the original image contains a user's face image. For example, taking a headshot image as an example, the present application can limit the removal of original images in which the number of face images is not one.

S303: Based on the size ratio and the location, select the original image that meets the preset conditions as the target image.

Furthermore, in order to determine whether the original image is an original image that meets preset conditions, the present application can determine the target image based on the size ratio of the facial image corresponding to each original image and the corresponding original image, as well as the position of the facial image in the original image.

It is understandable that for the user's image, the selection criteria can be determined by whether the face is clearly revealed when the image is taken, and whether the face is located in the center of the image. Therefore, the present application can further obtain the user's corresponding face image, and determine whether the corresponding image meets the requirements based on the size of the face image.

The present application can determine whether each original image is a target image based on whether the size ratio is within a preset standard interval. It should be noted that the present application does not specifically limit the standard interval, for example, it can be a 70%-80% interval, or a 75%-85% interval, etc.

S304, performing semantic segmentation on the target image to extract the background area in the target image.

For details, please refer to S202 in FIG. 2 , which will not be described in detail here.

S305, extracting a first feature corresponding to the background area, and determining the color type and color quantity corresponding to the background area based on the first feature.

S306: Determine a first analysis result based on the color type and the color quantity.

Among them, the first feature is the first color parameter, and the color type and color quantity corresponding to each background area are determined according to the first color parameter. Furthermore, for an original image containing a face image, too many color compositions in the background area, too bright or too dim color compositions will affect the beauty of the image. Therefore, the present application can determine the first analysis result based on the color type and color quantity corresponding to each background area, that is, whether the background area meets the conditions of the background area of the target image.

Furthermore, when the color types and quantity information included in the background area is detected, the present application can detect whether the number of colors included in the background exceeds a predetermined number and whether the color types include a predetermined color type. And according to the detection result, determine the first analysis result, that is, whether the original image corresponding to the background area meets the standard of the target image.

It should be noted that the present application does not specifically limit the types and number of colors in the background area. For example, after detecting that the background area contains white and black colors, it can be determined that the first analysis result is that the corresponding background area does not meet the background area of the target image. It can also be determined that after detecting that the number of colors in the background area exceeds 3 colors, it is determined that the first analysis result is that the original image corresponding to the background area does not meet the standard of the target image.

S307: extract at least one object region in the target image.

S308, extracting a second feature corresponding to the object region, and determining type information and/or attribute information of the object based on the second feature.

S309: Determine a second analysis result based on the type information and/or the attribute information.

S310: Determine a third analysis result of the target image based on the first analysis result and the second analysis result.

Among them, the second feature is the object parameter. Based on the preset neural network image detection model, the object parameters corresponding to the object area in each background area are extracted, and based on the object parameters, the type information and/or attribute information of the object is determined.

In a possible implementation, type information of each object is determined based on the object parameters, and a second analysis result is determined based on the type information of each object.

Furthermore, the present application can extract object feature parameters corresponding to the object region based on a preset neural network image detection model, and determine whether the object is contained therein. It is understandable that after determining that the object region is contained, the second analysis result, that is, whether the original image corresponding to the object region is consistent with the target image, is determined based on the type information of the object.

Furthermore, the neural network image detection model in the present application can be a convolutional neural network. Among them, convolutional neural networks (CNN) are a type of feedforward neural networks (Feedforward Neural Networks) that include convolution calculations and have a deep structure, and are one of the representative algorithms for deep learning. Convolutional neural networks have representation learning capabilities and can perform translation-invariant classification of input information according to their hierarchical structure. Thanks to the powerful feature representation capabilities of CNN (convolutional neural network) for images, it has achieved remarkable results in image classification, target detection, semantic segmentation and other fields.

It should be noted that before using the neural network image detection model to extract the object feature parameters corresponding to the object area, the present application can also first define the organ detection network architecture based on the cascaded region proposal network, region regression network and key point regression network structure using a deep convolutional neural network. In the deep convolutional neural network used, the region proposal network input is 16*16*3 image data, the network is composed of a full convolution architecture, and the output is the confidence of the object region proposal box and the rough vertex position; the region regression network input is 32*32*3 image data, the network is composed of a convolution and a fully connected architecture, and the output is the confidence of the background area and the precise vertex position; the key point regression network input is 64*64*3 image data, the network is composed of a convolution and a fully connected architecture, and the output is the confidence and position of the object shape information.

It should be noted that the present application does not specifically limit the method for determining the second analysis result based on the type information of each object image. For example, the present application can classify each object into teaching objects and non-teaching objects. It is understandable that teaching objects may include books, blackboards, tables and chairs, computers, etc. Non-teaching objects may include bowls and chopsticks, game consoles, water cups, etc.

Furthermore, the present application can determine that the second analysis result is that the background area corresponding to the object region meets the background area of the target image when the object region is detected to include a teaching object. And when the object region is detected to include a non-teaching object, the second analysis result is determined that the original image corresponding to the object region does not meet the standard of the target image.

In one possible implementation, attribute information of each object is determined based on object parameters, the attribute information including at least one of color information, quantity information, and size information; based on the type information and attribute information of each object, a second analysis result is determined, i.e., whether the original image corresponding to the object area meets the standards of the target image.

Furthermore, the present application can extract object feature parameters of each object region based on a preset neural network image detection model, and determine whether an object is contained therein. It is understandable that after determining that an object is contained, the second analysis result is determined through the attribute information of the object, that is, the original image corresponding to the object region meets the standard of the target image.

It should be noted that the present application does not specifically limit the method of determining the second analysis result based on the attribute information of each object. Among them, the attribute information can be information reflecting the color, quantity, and size of the object. It is understandable that for an original image containing a human body area, too many color compositions in the background area, too many objects placed, or the presence of overly large objects will affect the beauty of the image. Therefore, the present application can comprehensively determine the second analysis result based on the three parameters, that is, the original image corresponding to the object area meets the standards of the target image.

Furthermore, the present application can detect whether the color composition of the object exceeds a first quantity, whether the number of objects exceeds a second quantity, and whether the size exceeds a preset ratio of the corresponding background image when the object region includes at least one of color information, quantity information, and size information. If the corresponding conditions are met, a second analysis result is determined, that is, the original image corresponding to the object region meets the standard of the target image.

Furthermore, based on the first analysis result and the second analysis result, that is, according to the background area and the object area of the original image, a third analysis result is obtained, that is, it is determined whether the original image meets the standard of the target image.

When the scheme of the embodiment of the present application is executed, face recognition is performed on the original image, and based on the face recognition result of at least one original image, at least one target image is selected, the target image contains a human body area, and semantic segmentation is performed on the target image, that is, the foreground area and the background area of the target image are separated, the background area in the target image is extracted, and then the first feature corresponding to the background area is extracted, and the first analysis result corresponding to the background area is calculated based on the first feature. Through the present application, at least one target image can be selected from multiple original images, the background area can be extracted from the target image, and then the first feature corresponding to the background area can be calculated to obtain the first analysis result, thereby improving the accuracy and practicality of evaluating the image background in the related art.

FIG4 schematically shows a flow chart of an image analysis method according to an embodiment of the present application. As shown in FIG4 , the method includes:

S401, using a preset face recognition model, detecting whether the original image contains a face image.

S402, when it is determined that the original image contains a face image, obtain the size ratio of the face image to the corresponding original image and the position of the face image in the original image.

S403, based on the size ratio and the location, filter the original image that meets the preset conditions as the target image.

S404, performing semantic segmentation on the target image to extract the background area in the target image.

Generally, S401 to S404 may refer to 301 to S304 in FIG. 3 , and will not be described in detail here.

S405: extract a first feature corresponding to the background area, and calculate a matching degree between the first feature and a third feature corresponding to the human body area.

S406: Determine the first analysis result based on the matching degree.

Among them, the third feature is the second color parameter, and the color type and color quantity corresponding to the human body area are determined according to the second color parameter. Further, in order to ensure the beauty of the target image, the second color parameter corresponding to the human body area in the original image can be further determined first. Then, based on the color matching degree between the human body area and the background image, the first analysis result is determined, and the first analysis result is used to determine whether the original image corresponding to the background area meets the standard of the target image.

It is understandable that when the user wears brightly colored clothes, the corresponding background image should also be a background with richer color information, while when the user wears plainer clothes, the corresponding background image should also be a background with simpler color information.

S407: extract at least one object region in the target image.

S408, extracting a second feature corresponding to the object region, and determining type information and/or attribute information of the object based on the second feature.

S409: Determine a second analysis result based on the type information and/or the attribute information.

S410: Determine a third analysis result of the target image based on the first analysis result and the second analysis result.

Generally, S47 to S410 may refer to 307 to S310 in FIG. 3 , and will not be described in detail here.

When the scheme of the embodiment of the present application is executed, face recognition is performed on the original image, and based on the face recognition result of at least one original image, at least one target image is selected, the target image contains a human body area, and semantic segmentation is performed on the target image, that is, the foreground area and the background area of the target image are separated, the background area in the target image is extracted, and then the first feature corresponding to the background area is extracted, and the first analysis result corresponding to the background area is calculated based on the first feature. Through the present application, at least one target image can be selected from multiple original images, the background area can be extracted from the target image, and then the first feature corresponding to the background area can be calculated to obtain the first analysis result, thereby improving the accuracy and practicality of evaluating the image background in the related art.

In another embodiment of the present application, as shown in FIG5 , the present application further provides an image analysis device. The device includes a selection module 501, an extraction module 502, and a calculation module 503, wherein:

A selection module 501 is used to select at least one target image based on the face recognition result of at least one original image, wherein the target image contains a human body region;

An extraction module 502 is used to perform semantic segmentation on the target image and extract a background area in the target image;

The calculation module 503 is used to extract the first feature corresponding to the background area, and calculate the first analysis result of the background area based on the first feature.

Optionally, the device further comprises:

A second module 504 is used to identify at least one object in the target image and determine a second analysis result of the object based on the identification result;

The third module 505 is used to determine a third analysis result of the target image based on the first analysis result and the second analysis result.

Optionally, the second module 504 includes:

A first unit is used to extract at least one object region in the target image;

A second unit is used to extract a second feature corresponding to the object area, and determine type information and/or attribute information of the object based on the second feature;

The third unit is used to determine the second analysis result based on the type information and/or attribute information.

Optionally, the calculation module 503 includes:

A fourth unit, configured to determine the color type and the number of colors corresponding to the background area based on the first feature;

The fifth unit is used to determine the first analysis result based on the color type and the color quantity.

Optionally, the calculation module 503 includes:

a calculation unit, configured to calculate a matching degree between the first feature and a third feature corresponding to the human body region;

A determining unit is used to determine the first analysis result based on the matching degree.

Optionally, the selection module 501 includes:

A detection unit, used to detect whether the original image contains a face image by using a preset face recognition model;

a position determination unit, configured to, when determining that the original image contains a face image, obtain a size ratio of the face image to the corresponding original image and a position of the face image in the original image;

The screening unit is used to screen original images that meet preset conditions as the target images based on the size ratio and the location.

When the scheme of the embodiment of the present application is executed, face recognition is performed on the original image, and based on the face recognition result of at least one original image, at least one target image is selected, the target image contains a human body area, and semantic segmentation is performed on the target image, that is, the foreground area and the background area of the target image are separated, the background area in the target image is extracted, and then the first feature corresponding to the background area is extracted, and the first analysis result corresponding to the background area is calculated based on the first feature. Through the present application, at least one target image can be selected from multiple original images, the background area can be extracted from the target image, and then the first feature corresponding to the background area can be calculated to obtain the first analysis result, thereby improving the accuracy and practicality of evaluating the image background in the related art.

Fig. 6 is a block diagram of a logical structure of an electronic device according to an exemplary embodiment. For example, the electronic device 600 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

6 , an electronic device 600 may include one or more of the following components: a processor 601 and a memory 602 .

The processor 601 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 601 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 601 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 601 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 601 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

The memory 602 may include one or more computer-readable storage media, which may be non-transitory. The memory 602 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 602 is used to store at least one instruction, which is used to be executed by the processor 601 to implement the interactive special effects calibration method provided in the method embodiment of the present application.

In some embodiments, the electronic device 600 may further optionally include: a peripheral device interface 603 and at least one peripheral device. The processor 601, the memory 602 and the peripheral device interface 603 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 603 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 604, a touch display screen 605, a camera 606, an audio circuit 607, a positioning component 608 and a power supply 609.

The peripheral device interface 603 may be used to connect at least one peripheral device related to I/O (Input/Output) to the processor 601 and the memory 602. In some embodiments, the processor 601, the memory 602, and the peripheral device interface 603 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 601, the memory 602, and the peripheral device interface 603 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

The radio frequency circuit 604 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 604 communicates with the communication network and other communication devices through electromagnetic signals. The radio frequency circuit 604 converts the electrical signal into an electromagnetic signal for transmission, or converts the received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 604 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The radio frequency circuit 604 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: a metropolitan area network, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and/or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 604 may also include circuits related to NFC (NearField Communication), which is not limited in this application.

The display screen 605 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos and any combination thereof. When the display screen 605 is a touch display screen, the display screen 605 also has the ability to collect touch signals on the surface or above the surface of the display screen 605. The touch signal can be input to the processor 601 as a control signal for processing. At this time, the display screen 605 can also be used to provide virtual buttons and/or virtual keyboards, also known as soft buttons and/or soft keyboards. In some embodiments, the display screen 605 can be one, and the front panel of the electronic device 600 is set; in other embodiments, the display screen 605 can be at least two, which are respectively set on different surfaces of the electronic device 600 or are folded; in some other embodiments, the display screen 605 can be a flexible display screen, which is set on the curved surface or folded surface of the electronic device 600. Even, the display screen 605 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 605 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode, organic light-emitting diode).

The camera assembly 606 is used to capture images or videos. Optionally, the camera assembly 606 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize the panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 606 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

The audio circuit 607 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 601 for processing, or input them into the radio frequency circuit 604 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the electronic device 600. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 601 or the radio frequency circuit 604 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 607 may also include a headphone jack.

Positioning component 608 is used to locate the current geographic location of electronic device 600 to implement navigation or LBS (Location Based Service). Positioning component 608 can be a positioning component based on the US GPS (Global Positioning System), China's Beidou system, Russia's Grenas system or the European Union's Galileo system.

The power supply 609 is used to power various components in the electronic device 600. The power supply 609 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 609 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.

In some embodiments, the electronic device 600 further includes one or more sensors 610 , including but not limited to: an acceleration sensor 611 , a gyroscope sensor 612 , a pressure sensor 613 , a fingerprint sensor 614 , an optical sensor 615 , and a proximity sensor 616 .

The acceleration sensor 611 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the electronic device 600. For example, the acceleration sensor 611 can be used to detect the components of gravity acceleration on the three coordinate axes. The processor 601 can control the touch display screen 605 to display the user interface in a horizontal view or a vertical view according to the gravity acceleration signal collected by the acceleration sensor 611. The acceleration sensor 611 can also be used for collecting game or user motion data.

The gyro sensor 612 can detect the body direction and rotation angle of the electronic device 600, and the gyro sensor 612 can cooperate with the acceleration sensor 611 to collect the user's 3D actions on the electronic device 600. The processor 601 can implement the following functions based on the data collected by the gyro sensor 612: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

The pressure sensor 613 can be set on the side frame of the electronic device 600 and/or the lower layer of the touch display screen 605. When the pressure sensor 613 is set on the side frame of the electronic device 600, it can detect the user's holding signal of the electronic device 600, and the processor 601 performs left and right hand recognition or shortcut operation according to the holding signal collected by the pressure sensor 613. When the pressure sensor 613 is set on the lower layer of the touch display screen 605, the processor 601 controls the operability controls on the UI interface according to the user's pressure operation on the touch display screen 605. The operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

The fingerprint sensor 614 is used to collect the user's fingerprint, and the processor 601 identifies the user's identity based on the fingerprint collected by the fingerprint sensor 614, or the fingerprint sensor 614 identifies the user's identity based on the collected fingerprint. When the user's identity is identified as a trusted identity, the processor 601 authorizes the user to perform relevant sensitive operations, which include unlocking the screen, viewing encrypted information, downloading software, paying, and changing settings. The fingerprint sensor 614 can be set on the front, back, or side of the electronic device 600. When a physical button or a manufacturer logo is set on the electronic device 600, the fingerprint sensor 614 can be integrated with the physical button or the manufacturer logo.

The optical sensor 615 is used to collect the ambient light intensity. In one embodiment, the processor 601 can control the display brightness of the touch display screen 605 according to the ambient light intensity collected by the optical sensor 615. Specifically, when the ambient light intensity is high, the display brightness of the touch display screen 605 is increased; when the ambient light intensity is low, the display brightness of the touch display screen 605 is reduced. In another embodiment, the processor 601 can also dynamically adjust the shooting parameters of the camera assembly 606 according to the ambient light intensity collected by the optical sensor 615.

The proximity sensor 616, also called a distance sensor, is usually disposed on the front panel of the electronic device 600. The proximity sensor 616 is used to collect the distance between the user and the front of the electronic device 600. In one embodiment, when the proximity sensor 616 detects that the distance between the user and the front of the electronic device 600 is gradually decreasing, the processor 601 controls the touch display screen 605 to switch from the screen-on state to the screen-off state; when the proximity sensor 616 detects that the distance between the user and the front of the electronic device 600 is gradually increasing, the processor 601 controls the touch display screen 605 to switch from the screen-off state to the screen-on state.

Those skilled in the art will appreciate that the structure shown in FIG. 6 does not limit the electronic device 600 , and may include more or fewer components than shown, or combine certain components, or adopt a different component arrangement.

In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 604 including instructions, and the above instructions can be executed by the processor 620 of the electronic device 600 to complete the above method of selecting an image, the method comprising: extracting a background area image in the image to be selected, the image to be selected includes a user image and the background area image; based on a first color parameter and a clarity parameter, screening a first background image that meets a first preset condition in the background area image; based on an object image contained in each first background image, screening a second background image that meets a second preset condition in the first background image; and using the image to be selected corresponding to the second background image as a target image. Optionally, the above instructions can also be executed by the processor 620 of the electronic device 600 to complete other steps involved in the above exemplary embodiment. Optionally, the above instructions can also be executed by the processor 620 of the electronic device 600 to complete other steps involved in the above exemplary embodiment. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

In an exemplary embodiment, an application/computer program product is also provided, including one or more instructions, which can be executed by the processor 620 of the electronic device 600 to complete the above-mentioned image analysis method, which includes: based on the face recognition result of at least one original image, selecting at least one target image, the target image contains a human body area; performing semantic segmentation on the target image, extracting the background area in the target image; extracting the first feature corresponding to the background area, and calculating the first analysis result of the background area based on the first feature. Optionally, the above-mentioned instructions can also be executed by the processor 620 of the electronic device 600 to complete the other steps involved in the above-mentioned exemplary embodiment. Those skilled in the art will easily think of other embodiments of the present application after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variants, uses or adaptive changes of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the technical field that are not disclosed in the present application. The description and examples are only regarded as exemplary, and the true scope and spirit of the present application are indicated by the following claims.

It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims (7)

1. A method of image analysis, the method comprising:

Selecting at least one target image based on the face recognition result of the at least one original image, wherein the target image comprises a human body area;

carrying out semantic segmentation on the target image, and extracting a background area in the target image;

extracting a first feature corresponding to the background area, and calculating a first analysis result of the background area based on the first feature, wherein the first analysis result is used for representing whether color information of an original image corresponding to the background area meets the condition of a preset target image;

the calculating a first analysis result of the background region based on the first feature includes:

Calculating the matching degree of the first feature and a third feature corresponding to the human body region, wherein the first feature and the third feature comprise color parameters; determining the first analysis result based on the matching degree;

The method further comprises the steps of:

Extracting at least one object region in the target image;

Extracting a second feature corresponding to the object region, and determining type information and/or attribute information of the object based on the second feature;

Determining a second analysis result based on the type information and/or the attribute information, wherein the second analysis result is used for representing whether an original image corresponding to the object region meets the standard of the preset target image or not;

And determining a third analysis result of the target image based on the first analysis result and the second analysis result, wherein the third analysis result is used for representing whether the original image corresponding to the background area and the object area meets the standard of the preset target image.

2. The method of claim 1, wherein the attribute information includes at least one of color information, quantity information, and size information.

3. The method of claim 1, wherein the calculating the first analysis result of the background region based on the first feature comprises:

based on the first feature, determining the color type and the color number corresponding to the background area;

the first analysis result is determined based on the color category and the number of colors.

4. The method of claim 1, wherein the selecting at least one target image based on the face recognition result of the at least one original image comprises:

Detecting whether the original image contains a face image or not by using a preset face recognition model;

when the original image is determined to contain a face image, acquiring the size ratio of the face image to the corresponding original image and the position of the face image in the original image;

and screening an original image meeting preset conditions as the target image based on the size duty ratio and the position.

5. An image analysis apparatus, the apparatus comprising:

the recognition module is used for selecting at least one target image based on the face recognition result of the at least one original image, wherein the target image comprises a human body area;

The extraction module is used for carrying out semantic segmentation on the target image and extracting a background area in the target image;

the computing module is used for extracting first features corresponding to the background area, computing a first analysis result of the background area based on the first features, wherein the first analysis result is used for representing whether an original image corresponding to the background area meets the condition of a preset target image or not;

The computing module is specifically configured to compute a matching degree of the first feature and a third feature corresponding to the human body region, where the first feature and the third feature include color parameters;

determining the first analysis result based on the matching degree;

A second module for extracting at least one object region in the target image; extracting a second feature corresponding to the object region, and determining type information and/or attribute information of the object based on the second feature; determining a second analysis result based on the type information and/or attribute information;

And the third module is used for determining a third analysis result of the target image based on the first analysis result and the second analysis result, wherein the third analysis result is used for representing whether the original image corresponding to the background area and the object area meets the standard of the preset target image.

6. An electronic device, comprising:

a memory for storing executable instructions; and

A processor for displaying with the memory to execute the executable instructions to perform the operations of the image analysis method of any one of claims 1-4.

7. A computer readable storage medium storing computer readable instructions, wherein the instructions when executed perform the operations of the image analysis method of any one of claims 1-4.