CN117726963A - Picture data processing method and device, electronic equipment and medium - Google Patents

Abstract

The embodiments of the present application disclose a picture data processing method, device, electronic equipment and medium, which are applied in the field of multimedia technology. The method includes: obtaining the picture data to be detected, extracting the picture features of each picture frame from each picture frame, and obtaining the picture association based on the picture characteristics of each picture frame based on the adjacent picture frames of each picture frame. Processing to obtain associated screen features of the screen data, determining screen evaluation parameters of the screen data based on the associated screen features, and generating quality detection results for the screen data based on the screen evaluation parameters. By adopting the embodiments of the present application, the accuracy of the quality detection results of picture data can be improved.

Description

Screen data processing method, device, electronic equipment and media

Technical field

The present application relates to the field of multimedia technology, and in particular to a picture data processing method, device, electronic equipment and media.

Background technique

With the advent of the multimedia information age, various multimedia processing technologies emerge in endlessly. Therefore, the quality detection technology of picture data becomes increasingly important. For example, in the video recommendation scenario, the video data to be pushed to the user terminal can be combined with the quality detection results of the video data. At present, the existing quality detection methods are mainly carried out through manual annotation. The operators watch the picture fragments in the picture data and give scores to achieve the quality detection of the picture data. This method requires a lot of time and cost. The efficiency is low, resulting in inaccurate quality detection results of screen data.

Contents of the invention

Embodiments of the present application provide a screen data processing method, device, electronic equipment and media, which can improve the accuracy of the quality detection results of screen data.

On the one hand, embodiments of the present application provide a picture data processing method, which method includes:

Obtain the picture data to be detected; the picture data includes multiple picture frames arranged in time order in the picture data;

Extract the picture features of each picture frame from each picture frame respectively, and perform picture association processing on the picture features of each picture frame based on the adjacent picture frames of each picture frame to obtain the associated picture features of the picture data;

Determine the picture evaluation parameters of the picture data according to the associated picture characteristics; the picture evaluation parameters include one or more of the following: picture static evaluation parameters, picture dynamic evaluation parameters;

Generate quality inspection results for the picture data based on the picture evaluation parameters.

On the one hand, embodiments of the present application provide a screen data processing device, which includes:

The acquisition module is used to obtain the picture data to be detected; the picture data includes multiple picture frames arranged in time order in the picture data;

The processing module is used to extract the picture features of each picture frame from each picture frame, and perform picture association processing on the picture features of each picture frame based on the adjacent picture frames of each picture frame to obtain the picture data. associated picture features;

The processing module is also used to determine the picture evaluation parameters of the picture data according to the associated picture characteristics; the picture evaluation parameters include one or more of the following: static picture evaluation parameters and dynamic picture evaluation parameters.

The processing module is also used to generate quality detection results for the picture data according to the picture evaluation parameters.

On the one hand, embodiments of the present application provide an electronic device, which includes a processor and a memory, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions, Perform some or all of the steps in the above method.

On the one hand, embodiments of the present application provide a computer-readable storage medium. The computer-readable storage medium stores a computer program. The computer program includes program instructions. When the program instructions are executed by a processor, they are used to perform the above method. some or all of the steps.

Accordingly, according to one aspect of the present application, a computer program product or computer program is provided, which computer program product or computer program includes program instructions stored in a computer-readable storage medium. The processor of the computer device reads the program instructions from the computer-readable storage medium, and the processor executes the program instructions, so that the computer device performs some or all of the steps in the above method.

In the embodiment of the present application, the picture data to be detected can be obtained, the picture characteristics of each picture frame are extracted from each picture frame included in the picture data, and each picture frame is compared based on the adjacent picture frames of each picture frame. Perform picture association processing on the picture features to obtain the associated picture features of the picture data, and determine the picture evaluation parameters of the picture data based on the related picture characteristics; this can improve the richness of feature information covered by the related picture features, thereby obtaining accurate picture evaluation parameters; Generate quality inspection results for the picture data based on the picture evaluation parameters. The picture evaluation parameters may include one or more of the following: static picture evaluation parameters and dynamic picture evaluation parameters. The quality detection results of the screen data can be comprehensively determined through the screen evaluation parameters to improve the accuracy and reliability of the quality detection results.

Description of the drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present application, which are of great significance to this field. Ordinary technicians can also obtain other drawings based on these drawings without exerting creative work.

Figure 1 is a schematic diagram of a picture data processing scenario provided by an embodiment of the present application;

Figure 2 is a schematic flowchart of a screen data processing method provided by an embodiment of the present application;

Figure 3 is a schematic diagram of a scenario for determining quality detection results provided by an embodiment of the present application;

Figure 4 is a schematic flow chart of a screen data processing method provided by an embodiment of the present application;

Figure 5a is a schematic diagram of a scenario for determining picture evaluation parameters provided by an embodiment of the present application;

Figure 5b is a schematic diagram of a scenario for determining picture evaluation parameters provided by an embodiment of the present application;

Figure 6 is a schematic structural diagram of a picture data processing device provided by an embodiment of the present application;

FIG. 7 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of this application.

The screen data processing method proposed in the embodiment of the present application is implemented in an electronic device, and the electronic device may be a server or a terminal. Among them, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also provide cloud services, cloud computing, cloud functions, cloud storage, network services, cloud communications, and middleware. services, as well as cloud servers for basic cloud computing services such as big data and artificial intelligence platforms. The terminal can be a smartphone, a tablet, a laptop, a desktop computer, etc., but is not limited thereto.

The embodiment of the present application proposes a picture data processing method, which can obtain the picture data to be detected and determine multiple picture frames from the picture data. The multiple picture frames are arranged in sequence according to the time order in the picture data; respectively Extract the picture features of each picture frame from each picture frame, and perform picture association processing on the picture features of each picture frame based on the adjacent picture frames of each picture frame to obtain the associated picture features of the picture data. The associated picture features It contains richer feature information, so that more accurate picture evaluation parameters for the picture data can be determined based on the associated picture characteristics, and quality detection results for the picture data can be generated based on the picture evaluation parameters. The picture evaluation parameters may include one or more: static picture evaluation parameters and dynamic picture evaluation parameters. It can be understood that the quality detection results can be used to evaluate the picture quality of the picture data and can be used to measure the user's actual perception of the picture data. In addition, the picture static evaluation parameters are used to describe the static look and feel of the picture data, such as the picture quality look and feel of the picture data. Screen dynamic evaluation parameters are used to describe the dynamic look and feel of screen data, such as the look and feel of screen switching of screen data. That is to say, the picture evaluation parameters can be used to describe various types of user perceptions of the picture data. Therefore, the quality detection results determined comprehensively through the picture evaluation parameters can be more accurate and reliable.

Therefore, a schematic diagram of a picture data processing scenario proposed based on the picture data processing method can be shown in Figure 1. Figure 1 proposes a network architecture. The network architecture can include a business server and a user terminal cluster, where the user terminal The cluster may include one or more user terminals, and the number of user terminals in the user terminal cluster will not be limited here. Communication connections may exist between user terminals in a user terminal cluster. At the same time, any user terminal in the user terminal cluster can have a communication connection with the business server, so that each user terminal in the user terminal cluster can interact with the business server through the communication connection. The above-mentioned communication connection is not limited to a connection method. It can be connected directly or indirectly through wired communication, or directly or indirectly through wireless communication. It can also be connected through other methods, which is not limited in this application. In addition, it can be understood that the electronic device involved in the embodiment of the present application may be the service server shown in Figure 1, or any user terminal in the user terminal cluster shown in Figure 1.

It can be understood that the server can obtain the picture data uploaded by any user terminal, and determine the quality detection result of the picture data through the picture data processing method proposed in this application. For example, the server can divide the picture data into multiple picture frames in sequence according to the time order in the picture data, and extract the picture features of each picture frame in turn, and compare the picture data of each picture frame based on the adjacent picture frames of each picture frame. Features are processed by screen association to obtain the associated screen features of the screen data. The screen evaluation parameters of the screen data are determined based on the associated screen features. The quality detection results for the screen data can be generated based on the screen evaluation parameters, and the quality detection results and the screen data can be combined. The association is stored in the database.

Optionally, in some embodiments, the electronic device can execute the picture data processing method to determine the quality detection result of the picture data according to actual business requirements. The technical solution of this application can be applied to any picture data processing scenario. For example, in the scene of recommending picture data, when the electronic device determines the picture data to be recommended, it can combine the quality detection results of the picture data to determine the picture data to be recommended that has a better look and feel for the user.

Optionally, the data involved in this application, such as screen data, quality test results of screen data, etc., can be stored in a database, or can be stored in a blockchain, such as through a blockchain distributed system. This application does not limited.

It should be noted that in the specific implementation of the present application, when it comes to scenarios of obtaining user information and other related data, such as obtaining screen data uploaded by users, etc., user permission or consent needs to be obtained. That is, when the embodiments of this application are applied to specific products or technologies, the collection, use and processing of relevant user data comply with the relevant laws, regulations and standards of the relevant countries and regions. For example, prompt information can be sent in the form of an interactive interface to prompt the specific data that will be collected or obtained. Specifically, the type and content of these data can be prompted to the user through a list or other methods. Only users who receive permission to collect data on the interactive interface Only after the confirmation operation or instruction is completed, the relevant data will be further collected and processed.

It can be understood that the above scenarios are only examples and do not constitute a limitation on the application scenarios of the technical solutions provided by the embodiments of the present application. The technical solutions of the present application can also be applied to other scenarios. For example, those of ordinary skill in the art know that with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

Based on the above description, the embodiment of the present application proposes a picture data processing method, which can be executed by the above-mentioned electronic device. Please refer to Figure 2. Figure 2 is a schematic flow chart of a screen data processing method provided by an embodiment of the present application. As shown in Figure 2, the flow of the screen data processing method in this embodiment of the present application may include the following:

S201. Obtain the screen data to be detected.

Among them, the picture data can also be called video data, image data, etc., and can be real-time streaming picture data, picture data shot and uploaded by the target terminal (such as a user terminal), or based on a specified address (such as an address uploaded by a user terminal). Downloaded screen data, etc. The duration and type of screen data are not limited here. The picture data can be any picture data that requires quality inspection. The picture data may include a plurality of picture frames (also called video frames or image frames) arranged in time order in the picture data. For example, the plurality of picture frames may be called: the 1st frame, the 2nd frame...the i-1th frame, the i-th frame, the i+1th frame...the n-1th frame, the nth frame, and so on.

S202. Extract the picture features of each picture frame from each picture frame respectively, and perform picture association processing on the picture features of each picture frame based on the adjacent picture frames of each picture frame to obtain the associated picture features of the picture data. .

The picture characteristics of each picture frame include at least the characteristic information related to the picture content and the characteristic information related to the picture quality in the picture frame. It can be understood that picture features are used to characterize spatial features in picture frames. Optionally, extracting the picture features of each picture frame may include inputting each picture frame into a pre-trained first feature processing network in turn, and having the first feature processing network output the picture features of each picture frame. The first feature processing network may be trained based on a deep convolutional neural network. For example, the first feature processing network may adopt a visual geometry group (VisualGeometry Group, VGG) network structure or the like. The embodiments of this application do not limit the specific extraction method of picture features.

It can be understood that picture features only include feature information in corresponding picture frames. For picture data, the user's perception is not only reflected in each picture frame, but also reflected in the switching between each picture frame when the picture data is played. Therefore Picture association processing can be performed on adjacent picture frames to obtain the associated picture characteristics of the picture data. Specifically, the picture characteristics of each picture frame and the picture characteristics of adjacent picture frames of each picture frame are separately subjected to picture association processing to obtain the associated picture characteristics of each picture frame, based on the associated pictures of each picture frame. The characteristics determine the associated picture characteristics of the picture data. For example, the associated picture features of each picture frame may be used as the related picture features of the picture data.

The adjacent picture frames of each picture frame may refer to the previous picture frame of each picture frame, or may refer to the previous picture frame and the next picture frame of each picture frame. For example, the adjacent picture frame to the i-th picture frame can be the i-1th picture frame, or it can be the i-1th picture frame and the i+1th picture frame. It can be understood that when i=1, the i-1th picture frame can be a preset default picture frame (for example, it can be an initialized picture frame or a cover frame of picture data); when i=n, The i+1th picture frame may be a preset default picture frame. It can be understood that the associated picture characteristics of the picture data include the characteristic information of the picture frame itself, and also include the characteristic information covered when the picture frame is switched. Correlated picture features are used to characterize spatiotemporal features in picture frames.

In some embodiments, a picture frame is taken as an example to illustrate the process of picture association processing. For ease of understanding, take the i-th picture frame and the adjacent picture frame of the i-th picture frame as the i-1th picture frame as an example. Determining the associated picture characteristics of the i-th picture frame can be to obtain the i-th picture frame. The frame difference picture features between one picture frame and the i-th picture frame are subjected to feature fusion processing to obtain the associated picture features of the i-th picture frame. It can be understood that the associated picture characteristics include the characteristic information of the i-th picture frame itself, and also include the information between the i-1th picture frame and the i-th picture frame when switching from the i-1th picture frame to the i-th picture frame. Characteristic information brought about by screen changes.

Wherein, obtaining the frame difference picture feature between the i-1th picture frame and the i-th picture frame may be: obtaining the frame difference information between the i-1th picture frame and the i-th picture frame, and converting the i-th picture frame to the i-th picture frame. The frame difference information between the i-1 picture frame and the i-th picture frame is input into the first feature processing network, and the first feature processing network outputs frame difference picture features corresponding to the frame difference information. It can be understood that the pixel difference between the pixel information of the i-1th picture frame and the pixel information of the i-th picture frame can be used as the frame difference information. For example, a difference process may be performed on the pixel information of the i-th picture frame and the pixel information of the i-1th picture frame to obtain the frame difference information.

In addition, a specific method for performing feature fusion processing on the frame difference picture feature and the picture feature of the i-th picture frame may be to use the feature product of the frame difference picture feature and the picture feature of the i-th picture frame as the associated picture feature. Alternatively, the frame difference picture feature and the picture feature of the i-th picture frame may be input into the second feature processing network, and the second feature processing network outputs the associated picture feature. The second feature processing network may be a recurrent neural network (GRU, GatedRecurrent Unit).

Correspondingly, the adjacent picture frames of the i-th picture frame are the i-1th picture frame and the i+1th picture frame. For example, determining the associated picture characteristics of the i-th picture frame can be to obtain the i-1th picture frame. The first frame difference picture feature between the i-th picture frame and the i-th picture frame is obtained, and the second frame difference picture feature between the i-th picture frame and the i+1-th picture frame is obtained. For the first frame difference picture feature , perform feature fusion processing on the picture features of the i-th picture frame and the second frame difference picture features, and obtain the associated picture features of the i-th picture frame. It can be understood that the associated picture characteristics include the characteristic information of the i-th picture frame itself, and also include the information between the i-1th picture frame and the i-th picture frame when switching from the i-1th picture frame to the i-th picture frame. The characteristic information brought by the picture change, and the characteristics brought by the picture change between the i-th picture frame and the i+1-th picture frame when switching from the i-th picture frame to the i+1-th picture frame information. It can be understood that the difference processing can be performed on the pixel information of the i-th picture frame and the pixel information of the i-1th picture frame to obtain the first frame difference information; the pixel information of the i+1-th picture frame and The pixel information of the i-th picture frame is subjected to difference processing to obtain the second frame difference information.

In addition, a specific method for performing feature fusion processing on the first frame difference picture feature, the picture feature of the i-th picture frame, and the second frame difference picture feature may be to combine the first frame difference picture feature, the picture feature of the i-th picture frame The feature product with the second frame difference picture feature is used as the associated picture feature. Alternatively, the first frame difference picture feature, the picture feature of the i-th picture frame, and the second frame difference picture feature may be input into the second feature processing network, and the second feature processing network outputs the associated picture feature. The second feature processing network may be a recurrent neural network (GRU, Gated Recurrent Unit).

In some embodiments, determining the associated picture characteristics of the i-th picture frame may also include inputting the picture characteristics of the i-1th picture frame and the picture characteristics of the i-th picture frame into the second feature processing network, and the second feature processing network uses The feature processing network outputs associated picture features. Or, input the picture characteristics of the i-1th picture frame, the picture characteristics of the i-th picture frame and the picture characteristics of the i+1th picture frame into the second feature processing network, and the second feature processing network outputs the associated picture features .

Optionally, determining the associated picture characteristics of the i-th picture frame may also include determining the associated picture characteristics of the i-th picture frame based on the picture characteristics of the i-th picture frame and the associated picture characteristics of the i-1 th picture frame. For example, the picture features of the i-th picture frame and the associated picture features of the i-1th picture frame may be input into the second feature processing network to obtain the related picture features of the i-th picture frame. For example, input the picture characteristics of the first picture frame and the picture characteristics of the default picture into the second feature processing network to obtain the associated picture characteristics of the first picture frame; combine the picture characteristics of the second picture frame and the first picture frame The associated picture features are input into the second feature processing network to obtain the associated picture features of the second picture frame.

Optionally, a fully connected layer can be added between the first feature processing network and the second feature processing network. The picture features and/or frame difference picture features of the picture frame obtained through the first feature processing network can be obtained through the fully connected layer. The first layer performs dimensionality reduction processing, and then inputs the dimensionally reduced picture features into the second feature processing network, and the second feature processing network outputs the associated picture features.

S203. Determine the picture evaluation parameters of the picture data according to the associated picture characteristics.

Among them, the associated picture features of each picture frame can be subjected to feature fusion processing to obtain the target associated picture features of the picture data, and the target related picture features are input into the third feature processing network, and the third feature processing network outputs the picture of the picture data Evaluation parameters. Wherein, performing feature fusion processing on the associated picture features of each picture frame may be to perform feature splicing on the associated picture features of each picture frame, and use the spliced related picture features as the target related picture features. Alternatively, the feature product of the associated image features of each image feature may be used as the target associated image feature. The specific method of feature fusion is not limited here. The target associated picture features include feature information in the picture frame (i.e., static feature information) and also include feature information covered when the picture frame is switched (ie, dynamic feature information). The third feature processing network may be a fully connected layer of multi-layer connections. Through the third feature processing network, the target-related picture features can be predicted and output one or more picture evaluation parameters.

Therefore, the picture evaluation parameters may include one or more of the following: static picture evaluation parameters and dynamic picture evaluation parameters. Optionally, the picture static evaluation parameter is used to evaluate the quality score of the picture data in the static dimension. For example, it can be used to evaluate the quality score of the picture data in the picture quality dimension and the content dimension. The higher the picture static evaluation parameter, the better the picture data is. The clearer the image quality in the image quality dimension, and the richer the content in the content dimension. Further, the picture static evaluation parameters can also be subdivided into picture quality evaluation parameters and picture content evaluation parameters. The picture quality evaluation parameters are used to evaluate the quality score under the picture quality dimension of the picture data, and the picture content evaluation parameters are used to evaluate the picture data. Quality score under the content dimension. Optionally, the picture dynamic evaluation parameter is used to evaluate the quality score of the picture data in the dynamic dimension. For example, it can be used to evaluate the quality score of the picture data in the picture switching dimension and the picture timing dimension. The higher the picture dynamic evaluation parameter, the better the picture. The more coordinated the data switching is in the screen switching dimension, and the more stable the screen changes are in the screen timing dimension. Further, the screen dynamic evaluation parameters can also be subdivided into screen switching evaluation parameters and screen timing evaluation parameters. The screen switching evaluation parameters are used to evaluate the quality score of the screen data in the screen switching dimension. The screen timing evaluation parameters are used to evaluate the screen data. Quality score in the picture timing dimension. Therefore, the picture evaluation parameters can evaluate the user's static perception and dynamic perception during the playback of the picture data in various dimensions, so that the user's score of the picture data can be estimated in dimensions such as picture quality, content, and picture switching process.

S204. Generate quality detection results for the picture data according to the picture evaluation parameters.

Wherein, the average value corresponding to the parameter values of one or more picture evaluation parameters may be used as the quality detection result for the picture data. The higher the quality test result is, the better the quality of the picture data is, that is, the higher the user evaluation of the picture data is.

For example, as shown in Figure 3, Figure 3 is a schematic diagram of a scenario for determining quality detection results provided by an embodiment of the present application; wherein, the target detection model used to determine the quality detection results may include a first feature processing network, a fully connected layer , the second feature processing network and the third feature processing network; obtain picture data, which includes multiple picture frames (K1, K2,..., Kn), and input each picture frame into the first feature processing network in turn, Obtain the picture features of each picture frame, input the picture features of each picture frame into the fully connected layer in turn, obtain the reduced-dimensional picture features of each picture frame, and combine the reduced-dimensional picture features of each picture frame and the The associated picture features of adjacent picture frames are input into the second feature processing network to obtain the associated picture features of each picture frame. For example, the dimensionality reduction picture features of the i-th picture frame and the i-1th picture frame can be processed through the second feature processing network. The associated picture features output by the feature processing network are input into the second feature processing network to obtain the associated picture features of the i-th picture frame; feature fusion processing is performed on the associated picture features of each picture frame to obtain the target related picture features, and the target related picture features are obtained The third feature processing network is input, and the third feature processing network outputs picture evaluation parameters, that is, picture static evaluation parameters and picture dynamic evaluation parameters. It can be understood that the target detection model is trained through training sample data, and the training sample data is sample picture data and picture evaluation parameter labels corresponding to the sample picture data. For example, the picture evaluation parameter tags may include picture static evaluation parameter tags and picture dynamic evaluation parameter tags. For another example, the picture evaluation parameter tag may include the picture static evaluation parameter label and the picture dynamic evaluation parameter label, and the picture static evaluation parameter label specifically includes the picture quality evaluation parameter label and the picture content evaluation parameter label, and the picture dynamic evaluation parameter label specifically includes the picture switching. Evaluation parameter tag and picture timing evaluation parameter tag.

In the embodiment of the present application, the picture data to be detected can be obtained, the picture characteristics of each picture frame are extracted from each picture frame included in the picture data, and each picture frame is compared based on the adjacent picture frames of each picture frame. Perform picture association processing on the picture features to obtain the associated picture features of the picture data, and determine the picture evaluation parameters of the picture data based on the related picture characteristics; this can improve the richness of feature information covered by the related picture features, thereby obtaining accurate picture evaluation parameters; Generate quality inspection results for the picture data based on the picture evaluation parameters. The picture evaluation parameters may include one or more of the following: static picture evaluation parameters and dynamic picture evaluation parameters. The quality detection results of the screen data can be comprehensively determined through the screen evaluation parameters to improve the accuracy and reliability of the quality detection results.

Please refer to FIG. 4. FIG. 4 is a schematic flowchart of a picture data processing method provided by an embodiment of the present application. This method can be executed by the above-mentioned electronic device. As shown in Figure 4, the flow of the screen data processing method in the embodiment of the present application may include the following:

S401. Obtain the screen data to be detected.

S402. Extract the picture features of each picture frame from each picture frame respectively, and perform picture association processing on the picture features of each picture frame based on the adjacent picture frames of each picture frame to obtain the associated picture features of the picture data. . For the specific implementation of steps S401-S402, please refer to the relevant descriptions of the above embodiments, and will not be described again here.

S403. Determine the static characteristics of the picture and the dynamic characteristics of the picture from the associated picture characteristics, determine the static evaluation parameters of the picture based on the static characteristics of the picture, and determine the dynamic evaluation parameters of the picture based on the dynamic characteristics of the picture.

Among them, the picture evaluation parameters may include picture static evaluation parameters and picture dynamic evaluation parameters. The picture static characteristics used to determine the picture static evaluation parameters and the picture dynamic characteristics used to determine the picture dynamic evaluation parameters can be determined from the associated picture characteristics (also known as the picture dynamic characteristics). can be called picture motion features) to achieve feature layering for associated picture features. Since the features related to the static information of the picture and the features related to the dynamic information of the picture are separated, mutual interference between the features can be avoided, the accuracy of the determined picture evaluation parameters can be improved, and the use of large and complex structures can also be avoided. The model saves computational overhead and is more practical.

In some embodiments, determining the static characteristics of the picture and the dynamic characteristics of the picture from the associated picture features may be to sequentially input the associated picture features of multiple picture frames into the residual network, and use the residual network to extract the picture data from the multiple picture frames. Extract the picture residual features of multiple picture frames from the associated picture features, and use the picture residual features of multiple picture frames as picture static features, and use the feature difference between the related picture features and the picture static features as picture dynamic features feature. That is to say, by inputting the associated picture features of multiple picture frames into the residual network, the residual network can sequentially output the picture residual features related to the picture content information in each picture frame, and combine the picture residual features of each picture frame. The difference feature is used as the static feature of the picture. At this time, the associated picture feature of each picture frame can be differenced with the static picture feature of each picture frame to obtain the dynamic picture characteristics of each picture frame. Among them, the residual network can be, for example, Residual Network (ResNet), etc. The network structure of the residual network is not limited here.

In some embodiments, determining the picture static features and picture dynamic features of the picture data from the associated picture features may also include extracting each picture frame from the associated picture features of each picture frame based on the adjacent picture frames of each picture frame. For the correlation features of the picture frame, the correlation characteristics of each picture frame are used as the static characteristics of the picture, and the dynamic characteristics of the picture are determined from the associated picture features according to the static characteristics of the picture.

Among them, taking the adjacent picture frame of the i-th picture frame as the i-1th picture frame as an example, extracting the correlation features of the i-th picture frame may specifically include extracting the i-1th picture frame and the i-th picture frame. Between picture frames, the correlation features in the first direction, the correlation features in the second direction, and the correlation features in the third direction, according to the correlation features in the first direction, the correlation features in the second direction The correlation feature and the correlation feature in the third direction determine the correlation feature between the i-1th picture frame and the i-th picture frame. For example, the correlation feature between the i-1th picture frame and the i-th picture frame can be determined by using the correlation feature in the first direction, the correlation feature in the second direction, and the correlation feature in the third direction. The correlation features are spliced together, and the correlation features obtained by splicing are used as the correlation features between the i-1th picture frame and the i-th picture frame.

Therefore, the relevant feature information between the i-1th picture frame and the i-th picture frame can be determined from multiple dimensions, and the relevant feature information in the multiple dimensions can be used as the i-1th picture frame and the i-th picture frame. Correlation features between i picture frames. It can be understood that if the correlation characteristics between the i-1th picture frame and the i-th picture frame indicate that the i-1th picture frame and the i-th picture frame are more correlated, it means that the i-1th picture frame The less motion information and more static information generated when switching between frames and the i-th picture frame, so the correlation features between the i-1th picture frame and the i-th picture frame can be used as the i-th picture frame The static characteristics of each picture frame.

Wherein, the first direction may be the direction of the x-axis in the coordinate axis (which may also be called the x-direction in the spatial direction), and the correlation feature in the first direction may be to construct a first convolutional network for the first direction, as follows The first convolutional network performs feature convolution on the associated picture features of the i-1th picture frame and the associated picture features of the i-th picture frame to output correlation features in the first direction. Optionally, the first convolutional network may be constructed based on the Sobel operator (Sobel operator) in the x direction with a vector dimension of 3x3. The network structure of the first convolutional network is not limited here. For example, the first convolutional network can perform feature processing on the associated picture features of the i-1th picture frame and the associated picture features of the i-th picture frame to obtain the i-1th picture frame and the i-th picture frame. corresponding first joint picture features, and feature processing is performed on the first joint picture features to obtain correlation features in the first direction.

Wherein, the second direction may be the direction of the y-axis in the coordinate axis (which may also be called the y-direction in the spatial direction), and the correlation feature in the second direction may be to construct a second convolutional network for the second direction, as follows The second convolution network performs feature convolution on the associated picture features of the i-1th picture frame and the related picture features of the i-th picture frame to output correlation features in the second direction. Optionally, the second convolutional network can be constructed based on the sobel operator with a vector dimension of 3x3 in the y direction. The network structure of the first convolutional network is not limited here. For example, the second convolutional network can perform feature processing on the associated picture features of the i-1th picture frame and the associated picture features of the i-th picture frame to obtain the i-1th picture frame and the i-th picture frame. corresponding second joint picture features, and feature processing is performed on the second joint picture features to obtain correlation features in the second direction.

Among them, the third direction can be the direction of the z-axis in the coordinate axis (can also be called the time direction), and the correlation feature in the third direction can be to construct a third convolutional network for the third direction. The third convolutional network Feature convolution is performed on the associated picture features of the i-1th picture frame and the related picture features of the i-th picture frame to output correlation features in the third direction. Optionally, the third convolutional network can be constructed based on a convolutional network layer with a vector dimension of 2x1x1 and a vector with a value of [-1,1] as the convolution kernel. The network structure of the first convolutional network is not limited here. That is, the temporal feature deviation between two picture frames is extracted based on the third convolutional network. For example, the third convolutional network can perform feature processing on the associated picture features of the i-1th picture frame and the associated picture features of the i-th picture frame to obtain the i-1th picture frame and the i-th picture frame. corresponding third joint picture features, and feature processing is performed on the third joint picture features to obtain correlation features in the third direction.

Therefore, the static characteristics of each picture frame can be obtained through the above process. Correspondingly, if the adjacent picture frames of the i-th picture frame are the i-1th picture frame and the i+1th picture frame, the i-1th picture frame and the i-th picture frame can be determined respectively according to the above process. The correlation features between frames, the correlation features between the i-th picture frame and the i+1-th picture frame, and the correlation features between the i-1th picture frame and the i-th picture frame, the i-th picture frame The sum of the features of the correlation features between the picture frame and the i+1th picture frame is used as the picture static feature of the i-th picture frame.

In some embodiments, after determining the picture static characteristics of the picture data based on the associated picture characteristics of each picture frame, the picture dynamic characteristics may be determined based on the associated picture characteristics and the picture static characteristics of each picture frame. Specifically, the associated picture features of each picture frame are mapped through the feature mapping network into mapped picture features with the same dimensions as the picture static features, and the mapped picture features of each picture frame and the picture static features of each picture frame are separately mapped. Perform feature difference to obtain the picture dynamic characteristics of each picture frame, and use the picture dynamic characteristics of each picture frame as the picture dynamic characteristics of the picture data. Or, perform feature summation on the mapped picture features of each picture frame and the picture static features of each picture frame to obtain the summed picture features of each picture frame, and input the summed picture features of each picture frame into the convolution Neural layer, the convolutional neural layer outputs the dynamic characteristics of each picture frame.

In some embodiments, the third feature processing network may include a static evaluation network and a dynamic evaluation network. Determining the static evaluation parameters of the picture based on the static characteristics of the picture may include inputting the static characteristics of the picture into a static evaluation network, and having the static evaluation network output the static evaluation parameters of the picture. Determining the picture dynamic evaluation parameters based on the picture dynamic characteristics may include inputting the picture dynamic characteristics into a dynamic evaluation network, and having the dynamic evaluation network output the picture dynamic evaluation parameters.

Optionally, the parameter types of the picture static evaluation parameters may also include picture quality evaluation parameters and picture content evaluation parameters. The picture quality evaluation parameters can represent the results of quality evaluation of picture data from the picture quality dimension. The screen content evaluation parameter can represent the result of quality assessment of screen data from the screen content dimension. Therefore, the static characteristics of the picture for the picture quality evaluation parameters and the static characteristics of the picture for the picture content evaluation parameters can be determined respectively from the static characteristics of the picture, and the picture quality evaluation parameters and the picture content evaluation parameters are determined respectively through the corresponding static characteristics of the picture.

Optionally, determining the picture static evaluation parameters may specifically include determining the first picture area corresponding to the picture data, determining the local picture static characteristics from the picture static characteristics based on the first picture area corresponding to the picture data, and determining the local picture static characteristics according to the local picture static characteristics. Determine the picture quality evaluation parameters, perform feature dimensionality reduction processing on the static features of the picture, obtain the global static characteristics of the picture, and determine the picture content evaluation parameters based on the global static characteristics of the picture. It can be understood that in terms of picture quality, the picture quality of the local area of the picture data is similar to the picture quality of the global area. Therefore, the local picture static features can be intercepted from the picture static features of the picture data based on the first picture area of the picture data. , and use local static characteristics of the picture to determine the picture quality evaluation parameters, thereby reducing the scale of network parameters. For screen content, different areas of screen data contain different screen content information. Therefore, feature dimensionality reduction processing can be performed on the static features of the screen to obtain global static features of the screen, and the global static features of the screen can be used to determine the assessment of screen content. parameter. At this time, the global static characteristics of the picture include all content feature information in the picture data.

The first picture area corresponding to the picture data may refer to the first picture area for each picture frame. The first picture area for each picture frame is the same, and may be one or more. The first picture area corresponding to the picture frame may be a preset area. For example, the designated area in the middle of the picture frame, the designated area in the upper right corner, etc. Or it may be an area obtained by dividing the picture frame based on a preset number, and the divided preset number of picture areas can be spliced to obtain a complete picture frame. Or it may be an area determined by object recognition of the picture frame, such as an area selected by identifying a person in the picture frame, or an area selected by identifying a text in the picture frame. Each picture area is the same size. Each screen area may or may not have overlapping areas. The first picture area corresponding to each picture frame is the same, so determining the first picture area corresponding to one picture frame means determining the first picture area corresponding to each picture frame. When determining the first picture area corresponding to a picture frame, the picture area may be determined for a designated picture frame among multiple picture frames included in the picture data, such as the first picture frame, the last picture frame, or an arbitrarily selected picture frame. Picture frames etc. No limitation is made here.

It can be understood that the first picture area can be used to intercept the static characteristics of the picture to determine the local static characteristics of the picture. A first picture area can capture a local static feature of the picture. When there are multiple first picture areas, multiple intercepted local picture static features can be superimposed to serve as the final local picture static features. In addition, since the size of the picture data may be different from the size of the static features of the picture, a mapping relationship between the picture area and the interception area can be constructed. Based on the mapping relationship, the size of the interception area corresponding to the picture area can be determined, and the size of the interception area can be determined according to the size of the interception area. to intercept the static features of the picture. The mapping relationship may be set based on the correlation between the size of the picture data and the size of the static features of the picture. For example, the mapping relationship can be determined based on the pixel dimension of the picture data and the feature dimension of the static features of the picture.

In addition, the static evaluation network may include a first evaluation network and a second evaluation network. The first evaluation network may intercept the static features of the picture to obtain local static features of the picture, and perform feature processing on the local static features of the picture to obtain Picture quality evaluation parameters. The second evaluation network can perform dimensionality reduction processing on the static characteristics of the picture to obtain the global static characteristics of the picture, and perform feature processing on the global static characteristics of the picture to obtain the picture content evaluation parameters.

Correspondingly, the parameter types of the picture dynamic evaluation parameters may include picture switching evaluation parameters and picture timing evaluation parameters. The screen switching evaluation parameter can represent the result of quality evaluation of the screen data from the screen switching dimension. This picture switching dimension can measure the picture aesthetic elements and shooting skills of the picture data. The picture timing evaluation parameter can represent the result of quality evaluation from the picture data from the picture timing dimension. Therefore, the static characteristics of the picture for the picture switching evaluation parameters and the static characteristics of the picture for the timing evaluation parameters can be determined from the dynamic characteristics of the picture, and the picture switching evaluation parameters and the picture timing evaluation parameters are determined respectively through the corresponding static characteristics of the picture.

Optionally, determining the picture dynamic evaluation parameters may specifically include determining the second picture area corresponding to the picture data, determining the local picture dynamic characteristics from the picture dynamic characteristics based on the second picture area corresponding to the picture data, and determining the local picture dynamic characteristics according to the local picture dynamic characteristics. Determine the picture timing evaluation parameters, perform feature dimensionality reduction processing on the picture dynamic characteristics, obtain the global picture dynamic characteristics, and determine the picture switching evaluation parameters based on the global picture dynamic characteristics. It can be understood that for the picture timing, the picture timing stability of the local area of the picture data is similar to the picture timing stability of the global area. Therefore, the local area can be intercepted from the picture dynamic characteristics of the picture data based on the second picture area of the picture data. The dynamic characteristics of the picture are determined, and the local dynamic characteristics of the picture are used to determine the picture timing evaluation parameters, thereby reducing the scale of network parameters. For screen switching, different areas of screen data contain different screen content information, so the effect of screen switching is also different. Therefore, feature dimensionality reduction processing can be performed on the dynamic features of the screen to obtain global dynamic features of the screen. , and use the global dynamic characteristics of the picture to determine the picture switching evaluation parameters. At this time, the global dynamic characteristics of the picture include the switching characteristic information of all contents in the picture data.

It can be understood that the second picture area is the same as the first picture area. The process and principle of intercepting the dynamic characteristics of the picture to obtain local dynamic characteristics of the picture can be referred to the process and principle of intercepting the static characteristics of the picture to obtain local static characteristics of the picture. In addition, the dynamic evaluation network may include a third evaluation network and a fourth evaluation network. The third evaluation network may intercept and process the dynamic characteristics of the picture to obtain local dynamic characteristics of the picture, and perform feature processing on the local dynamic characteristics of the picture to obtain Screen timing evaluation parameters. The fourth evaluation network can perform dimensionality reduction processing on the dynamic characteristics of the picture to obtain the global dynamic characteristics of the picture, and perform feature processing on the dynamic characteristics of the global picture to obtain the picture switching evaluation parameters.

Therefore, through the above description, embodiments of the present application provide a target detection model through which the picture evaluation parameters of the picture data can be determined. The target detection model may include a first feature processing network, a second feature processing network, a feature processing network, and a feature processing network. Hierarchical network and tertiary feature processing network. The feature layered network may include a residual network or multiple convolutional networks (such as the first convolutional network, the second convolutional network and the third convolutional network mentioned above). The third feature processing network may include a dynamic evaluation network and a static evaluation network, and the static evaluation network may include the above-mentioned first evaluation network and the second evaluation network, and the dynamic evaluation network may include the above-mentioned third evaluation network and the fourth evaluation network. Optionally, a fully connected layer may also be included between the first feature processing network and the second feature processing network in the target detection model. The target detection models mentioned above have been trained in advance through the training sample data and the image evaluation parameter labels of the training sample data.

For example, as shown in Figure 5a-Figure 5b, Figure 5a-Figure 5b is a schematic diagram of a scene for determining picture evaluation parameters provided by an embodiment of the present application; wherein the target detection model is shown in Figure 5a, and the process of determining the picture evaluation parameters can be Yes: perform frame processing on the picture data to obtain multiple picture frames included in the picture data, input the multiple picture frames in sequence to the first feature processing network to obtain the picture characteristics of each picture frame; input the picture characteristics of each picture frame The second feature processing network performs feature correlation processing on the picture features of each picture frame and the associated picture features of adjacent picture frames of each picture frame to output the associated picture features of each picture frame; The associated picture features of each picture frame are input into the feature hierarchical network. Multiple convolutional networks in the feature segmentation network perform feature layering processing on the associated picture features of the picture data to obtain the picture dynamic characteristics and picture static characteristics of the picture data. ; Output the static characteristics of the picture to the first evaluation network in the third feature processing network to obtain local static characteristics of the picture, and obtain the picture quality evaluation parameters based on the local static characteristics of the picture; output the static characteristics of the picture to the third feature processing network The second evaluation network obtains the global static characteristics of the picture, and obtains the picture content evaluation parameters based on the global static characteristics of the picture; outputs the dynamic characteristics of the picture to the third evaluation network in the third feature processing network, obtains the local dynamic characteristics of the picture, and The picture timing evaluation parameters are obtained based on the local picture dynamic characteristics; the picture dynamic characteristics are output to the fourth evaluation network in the third feature processing network to obtain the global picture dynamic characteristics, and the picture switching evaluation parameters are obtained based on the global picture dynamic characteristics.

In addition, the local static features of the picture are determined as shown in Figure 5b. Taking the first picture area corresponding to the picture data including area 1 and area 2 as an example, the picture can be determined based on the size of the picture area of the picture frame and the characteristic size of the local static features. The mapping relationship between the area and the interception area can determine the interception area 1 corresponding to area 1 and the interception area 2 corresponding to area 2. Based on the interception area 1, we can intercept the static characteristics of the picture to obtain the local static characteristics of the picture. Based on the interception area 1, we can obtain the local static characteristics of the picture. The static features of the screen are intercepted to obtain the local static features of the screen 1, and the static features of the screen are intercepted based on the interception area 2 to obtain the local static features of the screen 2. The local static features of the screen 1 and the local static features of the screen 2 are superimposed to obtain the screen. The final local picture static characteristics of the data. It can be understood that for each picture frame, the local picture static features corresponding to each picture frame can be obtained according to the foregoing process.

In some embodiments, for different picture data, the user may not necessarily measure the picture data from all evaluation dimensions when watching. For example, for landscape picture data, users may mainly care about picture quality and picture timing. For daily screen data, users may be mainly concerned about screen content and screen switching. Therefore, it is also possible to perform scene detection on the picture data to obtain the picture scene information of the picture data, and determine the parameter information of the picture evaluation parameters to be obtained according to the picture scene information. The parameter information indicates the type of picture evaluation parameters that should be obtained by the picture data. The parameter value of at least one picture evaluation parameter indicated by the parameter information may be determined according to the associated picture characteristics. For example, if the parameter information indicates the picture quality evaluation parameters and the picture timing evaluation parameters, the static characteristics of the picture for the picture quality evaluation parameters and the dynamic characteristics of the picture for the picture timing evaluation parameters may be determined based on the associated picture information, and the picture quality evaluation parameters may be determined based on the associated picture information. The parameter value of the picture quality evaluation parameter is determined based on the static characteristics of the picture and the parameter value of the picture timing evaluation parameter is determined based on the dynamic characteristics of the picture for the picture timing evaluation parameter. Among them, the picture scene information can represent the picture type of the picture data, such as landscape type, daily type, etc. Scene detection may be performed on the picture data through a scene detection model to output picture scene information. The scene detection model is trained based on sample picture data and picture scene labels. In addition, the correspondence between picture scene information and parameter information can be set based on experience values.

S404. Generate quality detection results for the picture data according to the picture static evaluation parameters and the picture dynamic evaluation parameters.

Wherein, the average value corresponding to the parameter value of one or more static picture evaluation parameters and the parameter value of one or more dynamic picture evaluation parameters may be used as the quality detection result. Alternatively, the parameter values of the static picture evaluation parameters and the parameter values of the dynamic picture evaluation parameters can also be weighted and summed to obtain the quality detection result.

Optionally, there may be multiple picture data, and the average value corresponding to the parameter value of the picture evaluation parameter of each picture data in the plurality of picture data may be used as the quality detection result for each picture data. The quality detection results can be used to determine the picture data to be recommended. For example, a target number of screen data can be selected from the multiple screen data in descending order according to the quality detection results of the multiple screen data, and the target number of screen data can be determined as the screen data to be recommended, and the target number of screen data can be selected. The screen data to be recommended is pushed to the target terminal.

For example, in the search scenario of screen data, multiple matching screen data can be obtained from the database based on the user's search terms, and based on the matching of the multiple screen data and the search terms and the multiple screen data The quality inspection results are comprehensively determined from multiple screen data to determine the screen data to be recommended to the user.

In the embodiment of the present application, the picture data to be detected can be obtained, the picture characteristics of each picture frame are extracted from each picture frame included in the picture data, and each picture frame is compared based on the adjacent picture frames of each picture frame. Perform picture association processing on the picture characteristics to obtain the associated picture characteristics of the picture data, determine the picture static characteristics and picture dynamic characteristics of the picture data from the associated picture characteristics, determine the picture static evaluation parameters according to the picture static characteristics, and determine the picture static characteristics according to the picture dynamic characteristics Determine the picture dynamic evaluation parameters; this can improve the feature information richness covered by the associated picture features, thereby obtaining accurate picture evaluation parameters, and determine the picture evaluation parameters of different dimensions according to the characteristics of different dimensions; according to the picture static evaluation parameters and picture dynamics Evaluating parameters generates quality inspection results for screen data. The quality detection results of the screen data can be comprehensively determined through the screen evaluation parameters to improve the accuracy and reliability of the quality detection results.

Please refer to FIG. 6 , which is a schematic structural diagram of a picture data processing device provided by the present application. It should be noted that the screen data processing device shown in Figure 6 is used to execute the methods of the embodiments shown in Figures 2 and 4 of the present application. For ease of explanation, only the parts related to the embodiments of the present application are shown. Specifically, Technical details are not disclosed, and reference is made to the embodiments shown in Figures 2 and 4 of the present application. The picture data processing device 600 may include: an acquisition module 601 and a processing module 602. in:

The acquisition module 601 is used to obtain the picture data to be detected; the picture data includes multiple picture frames arranged in time order in the picture data;

The processing module 602 is used to extract the picture features of each picture frame from each picture frame, and perform picture association processing on the picture features of each picture frame based on the adjacent picture frames of each picture frame to obtain picture data. associated picture features;

The processing module 602 is also used to determine the picture evaluation parameters of the picture data according to the associated picture characteristics; the picture evaluation parameters include one or more of the following: static picture evaluation parameters and dynamic picture evaluation parameters;

The processing module 602 is also used to generate quality detection results for the picture data according to the picture evaluation parameters.

In some embodiments, when the processing module 602 is used to perform picture association processing on the picture features of each picture frame based on the adjacent picture frames of each picture frame to obtain the associated picture characteristics of the picture data, it is specifically used to:

Perform picture correlation processing on the picture characteristics of each picture frame and the picture characteristics of adjacent picture frames of each picture frame to obtain the associated picture characteristics of each picture frame;

The associated picture characteristics of the picture data are determined based on the associated picture characteristics of each picture frame.

In some embodiments, the processing module 602 is also used to:

Perform scene detection on the picture data to obtain the picture scene information of the picture data;

Determine the parameter information of the picture evaluation parameters to be obtained according to the picture scene information;

Determine the picture evaluation parameters of the picture data based on the associated picture characteristics, including:

A parameter value of at least one picture evaluation parameter indicated by the parameter information is determined according to the associated picture characteristics.

In some embodiments, the picture evaluation parameters include picture static evaluation parameters and picture dynamic evaluation parameters;

When the processing module 602 is used to determine the picture evaluation parameters of the picture data according to the associated picture characteristics, it is specifically used to:

Determine the static characteristics of the picture and the dynamic characteristics of the picture from the associated picture characteristics;

The static evaluation parameters of the picture are determined according to the static characteristics of the picture, and the dynamic evaluation parameters of the picture are determined according to the dynamic characteristics of the picture.

In some embodiments, the picture static evaluation parameters include: picture quality evaluation parameters and picture content evaluation parameters; when the processing module 602 is used to determine the picture static evaluation parameters according to the picture static characteristics, it is specifically used to:

Determine the first picture area corresponding to the picture data, and determine the local picture static characteristics from the picture static characteristics based on the first picture area corresponding to the picture data;

Determine picture quality evaluation parameters based on local static characteristics of the picture;

Perform feature dimensionality reduction on the static features of the picture to obtain global static features of the picture;

Screen content evaluation parameters are determined based on global static characteristics of the screen.

In some embodiments, the picture dynamic evaluation parameters include: picture switching evaluation parameters and picture timing evaluation parameters; when the processing module 602 is used to determine the picture dynamic evaluation parameters according to the picture dynamic characteristics, it is specifically used to:

Determine the second picture area corresponding to the picture data, and determine the local picture dynamic characteristics from the picture dynamic characteristics based on the second picture area corresponding to the picture data;

Determine picture timing evaluation parameters based on local picture dynamic characteristics;

Perform feature dimensionality reduction on the dynamic features of the picture to obtain global dynamic characteristics of the picture;

The screen switching evaluation parameters are determined according to the global dynamic characteristics of the screen.

In some embodiments, there is multiple picture data; when the processing module 602 is used to generate quality detection results for the picture data according to the picture evaluation parameters, it is specifically used to:

The average value corresponding to the parameter value of the picture evaluation parameter of each picture data in the plurality of picture data is used as the quality detection result for each picture data;

The processing module 602 is also used to:

According to the order of the quality detection results of the multiple screen data from large to small, a target number of screen data are selected from the multiple screen data, and the target number of screen data are determined as the screen data to be recommended;

Push the screen data to be recommended to the target terminal.

In the embodiment of the present application, the acquisition module obtains the picture data to be detected; the processing module extracts the picture characteristics of each picture frame from each picture frame, and compares the characteristics of each picture frame based on the adjacent picture frames of each picture frame. The picture characteristics are processed to obtain picture correlation, and the associated picture characteristics of the picture data are obtained; the processing module determines the picture evaluation parameters of the picture data based on the associated picture characteristics; the processing module generates quality detection results for the picture data based on the picture evaluation parameters. Through the above device, the richness of feature information covered by the associated picture features can be improved, thereby obtaining accurate picture evaluation parameters. The quality detection results of the picture data can be comprehensively determined through the picture evaluation parameters to improve the accuracy and reliability of the quality detection results. .

Please refer to FIG. 7 , which is a schematic structural diagram of an electronic device provided by an embodiment of the present application. As shown in FIG. 7 , the electronic device 700 includes: at least one processor 701 and a memory 702 . Optionally, the electronic device may also include a network interface. Among them, the processor 701, the memory 702 and the network interface can exchange data. The network interface is controlled by the processor 701 and is used to send and receive messages. The memory 702 is used to store computer programs. The computer program includes program instructions, and the processor 701 is used to execute Memory 702 stores program instructions. Wherein, the processor 701 is configured to call program instructions to execute the above method.

The memory 702 may include volatile memory (volatile memory), such as random-access memory (RAM); the memory 702 may also include non-volatile memory (non-volatile memory), such as flash memory (flash memory). ), solid-state drive (SSD), etc.; the memory 702 may also include a combination of the above types of memory.

The processor 701 may be a central processing unit (CPU). In one embodiment, the processor 701 may also be a graphics processing unit (GPU). The processor 701 may also be a combination of a CPU and a GPU.

In a possible implementation, the memory 702 is used to store program instructions, and the processor 701 can call the program instructions to perform the following steps:

Obtain the picture data to be detected; the picture data includes multiple picture frames arranged in time order in the picture data;

Extract the picture features of each picture frame from each picture frame respectively, and perform picture association processing on the picture features of each picture frame based on the adjacent picture frames of each picture frame to obtain the associated picture features of the picture data;

Determine the picture evaluation parameters of the picture data according to the associated picture characteristics; the picture evaluation parameters include one or more of the following: picture static evaluation parameters, picture dynamic evaluation parameters;

Generate quality inspection results for the picture data based on the picture evaluation parameters.

In some embodiments, when the processor 701 is used to perform picture association processing on the picture characteristics of each picture frame based on the adjacent picture frames of each picture frame to obtain the associated picture characteristics of the picture data, it is specifically used to:

Perform picture correlation processing on the picture characteristics of each picture frame and the picture characteristics of adjacent picture frames of each picture frame to obtain the associated picture characteristics of each picture frame;

The associated picture characteristics of the picture data are determined based on the associated picture characteristics of each picture frame.

In some embodiments, processor 701 is also used to:

Perform scene detection on the picture data to obtain the picture scene information of the picture data;

Determine the parameter information of the picture evaluation parameters to be obtained according to the picture scene information;

Determine the picture evaluation parameters of the picture data based on the associated picture characteristics, including:

A parameter value of at least one picture evaluation parameter indicated by the parameter information is determined according to the associated picture characteristics.

In some embodiments, the picture evaluation parameters include picture static evaluation parameters and picture dynamic evaluation parameters;

When the processor 701 is used to determine the picture evaluation parameters of the picture data according to the associated picture characteristics, it is specifically used to:

Determine the static characteristics of the picture and the dynamic characteristics of the picture from the associated picture characteristics;

The static evaluation parameters of the picture are determined according to the static characteristics of the picture, and the dynamic evaluation parameters of the picture are determined according to the dynamic characteristics of the picture.

In some embodiments, the picture static evaluation parameters include: picture quality evaluation parameters and picture content evaluation parameters; when the processor 701 is used to determine the picture static evaluation parameters according to the picture static characteristics, it is specifically used to:

Determine the first picture area corresponding to the picture data, and determine the local picture static characteristics from the picture static characteristics based on the first picture area corresponding to the picture data;

Determine picture quality evaluation parameters based on local static characteristics of the picture;

Perform feature dimensionality reduction on the static features of the picture to obtain global static features of the picture;

Screen content evaluation parameters are determined based on global static characteristics of the screen.

In some embodiments, the picture dynamic evaluation parameters include: picture switching evaluation parameters and picture timing evaluation parameters; when the processor 701 is used to determine the picture dynamic evaluation parameters according to the picture dynamic characteristics, it is specifically used to:

Determine the second picture area corresponding to the picture data, and determine the local picture dynamic characteristics from the picture dynamic characteristics based on the second picture area corresponding to the picture data;

Determine picture timing evaluation parameters based on local picture dynamic characteristics;

Perform feature dimensionality reduction on the dynamic features of the picture to obtain global dynamic characteristics of the picture;

The screen switching evaluation parameters are determined according to the global dynamic characteristics of the screen.

In some embodiments, there is multiple picture data; when the processor 701 is used to generate quality detection results for the picture data according to the picture evaluation parameters, it is specifically used to:

The average value corresponding to the parameter value of the picture evaluation parameter of each picture data in the plurality of picture data is used as the quality detection result for each picture data;

Processor 701 is also used for:

According to the order of the quality detection results of the multiple screen data from large to small, a target number of screen data are selected from the multiple screen data, and the target number of screen data are determined as the screen data to be recommended;

Push the screen data to be recommended to the target terminal.

In specific implementation, the devices, processors, memories, etc. described in the embodiments of this application can execute the implementation described in the above method embodiments, or can also execute the implementation described in the embodiments of this application, which will not be described again here.

The embodiment of the present application also provides a computer (readable) storage medium. The computer storage medium stores a computer program. The computer program includes program instructions. When the program instructions are executed by the processor, the processor can execute the steps in the above method embodiments. Perform some or all of the steps. Optionally, the computer storage medium may be volatile or non-volatile. The computer-readable storage medium may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, at least one application required for a function, etc.; the storage data area may store data generated according to the use of the blockchain node. Created data, etc.

Embodiments of the present application provide a computer program product. The computer program product may include a computer program. When the computer program is executed by a processor, it may implement some or all of the steps in the above method, which will not be described again here.

The "plurality" mentioned in this article means two or more than two. "And/or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and/or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the related objects are in an "or" relationship.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer storage medium, and the computer storage medium can be a computer. In a readable storage medium, when the program is executed, it may include the processes of the above-mentioned method embodiments. The storage medium may be a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM) or a random access memory (Random Access Memory, RAM), etc.

What is disclosed above is only some of the embodiments of the present application. Of course, it cannot be used to limit the scope of rights of the present application. Those of ordinary skill in the art can understand all or part of the processes for implementing the above embodiments and make decisions according to the claims of the present application. Equivalent changes are still within the scope of this application.

Claims (10)

1. A picture data processing method, characterized in that the method comprises:

acquiring picture data to be detected; the picture data includes a plurality of picture frames arranged in a time order in the picture data;

extracting picture characteristics of each picture frame from each picture frame respectively, and carrying out picture association processing on the picture characteristics of each picture frame based on adjacent picture frames of each picture frame to obtain associated picture characteristics of the picture data;

determining picture evaluation parameters of the picture data according to the associated picture characteristics; the picture evaluation parameters include one or more of the following: picture static evaluation parameters and picture dynamic evaluation parameters;

and generating a quality detection result for the picture data according to the picture evaluation parameters.

2. The method according to claim 1, wherein said performing a picture correlation process on the picture features of each picture frame based on the neighboring picture frames of each picture frame to obtain the correlated picture features of the picture data comprises:

Respectively carrying out picture association processing on the picture characteristics of each picture frame and the picture characteristics of the adjacent picture frames of each picture frame to obtain the associated picture characteristics of each picture frame;

and determining the associated picture characteristics of the picture data based on the associated picture characteristics of each picture frame.

3. The method according to claim 1, wherein the method further comprises:

performing scene detection on the picture data to obtain picture scene information of the picture data;

determining parameter information of picture evaluation parameters to be acquired according to the picture scene information;

the determining the picture evaluation parameters of the picture data according to the associated picture features comprises:

and determining a parameter value of at least one picture evaluation parameter indicated by the parameter information according to the associated picture characteristics.

4. The method of claim 1, wherein the picture evaluation parameters include the picture static evaluation parameters and the picture dynamic evaluation parameters;

the determining the picture evaluation parameters of the picture data according to the associated picture features comprises:

determining picture static features and picture dynamic features of the picture data from the associated picture features;

And determining the picture static evaluation parameters according to the picture static characteristics, and determining the picture dynamic evaluation parameters according to the picture dynamic characteristics.

5. The method of claim 4, wherein the visual static assessment parameters comprise: picture quality evaluation parameters, picture content evaluation parameters; the determining the picture static evaluation parameter according to the picture static characteristics comprises the following steps:

determining a first picture area corresponding to the picture data, and determining local picture static features from the picture static features based on the first picture area corresponding to the picture data;

determining the picture quality evaluation parameters according to the local picture static characteristics;

performing feature dimension reduction processing on the picture static features to obtain global picture static features;

and determining the picture content evaluation parameters according to the global picture static characteristics.

6. The method of claim 4, wherein the visual dynamic assessment parameters comprise: picture switching evaluation parameters, picture timing evaluation parameters; the determining the picture dynamic evaluation parameter according to the picture dynamic characteristics comprises the following steps:

Determining a second picture area corresponding to the picture data, and determining local picture dynamic characteristics from the picture dynamic characteristics based on the second picture area corresponding to the picture data;

determining the picture time sequence evaluation parameters according to the local picture dynamic characteristics;

performing feature dimension reduction processing on the picture dynamic features to obtain global picture dynamic features;

and determining the picture switching evaluation parameters according to the global picture dynamic characteristics.

7. The method of any of claims 1-6, wherein the picture data is a plurality of; the generating a quality detection result for the picture data according to the picture evaluation parameter includes:

taking an average value corresponding to a parameter value of a picture evaluation parameter of each picture data in a plurality of picture data as a quality detection result for each picture data;

the method further comprises the steps of:

sequentially selecting a target number of picture data from the plurality of picture data according to the sequence from the large to the small of the quality detection results of the plurality of picture data, and determining the target number of picture data as picture data to be recommended;

Pushing the picture data to be recommended to a target terminal.

8. A picture data processing apparatus, characterized in that the apparatus comprises:

the acquisition module is used for acquiring picture data to be detected; the picture data includes a plurality of picture frames arranged in a time order in the picture data;

the processing module is used for extracting the picture characteristics of each picture frame from each picture frame respectively, and carrying out picture association processing on the picture characteristics of each picture frame based on the adjacent picture frames of each picture frame to obtain the associated picture characteristics of the picture data;

the processing module is further used for determining picture evaluation parameters of the picture data according to the associated picture characteristics; the picture evaluation parameters include one or more of the following: picture static evaluation parameters and picture dynamic evaluation parameters;

the processing module is further configured to generate a quality detection result for the picture data according to the picture evaluation parameter.

9. An electronic device comprising a processor and a memory, wherein the memory is configured to store a computer program comprising program instructions, the processor being configured to invoke the program instructions to perform the method of any of claims 1-7.

10. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program comprising program instructions which, when executed by a processor, cause the processor to perform the method of any of claims 1-7.