CN113971748B - Image processing method, device, equipment and computer readable storage medium - Google Patents

Abstract

The embodiment of the application provides an image processing method, device, equipment and a computer readable storage medium, wherein the method comprises the steps of extracting characteristics of an image to be processed to obtain direction characteristics and image characteristics of the image to be processed, identifying the direction characteristics to obtain categories of the image to be processed, carrying out image reconstruction processing on the image to be processed according to the categories and the image characteristics to obtain a processed image, and outputting the processed image. According to the application, the image to be processed can be accurately reconstructed according to the accurate category, and the processed image meeting the processing requirement of the user can be obtained.

Description

Image processing method, device, equipment and computer readable storage medium

Technical Field

Embodiments of the present application relate to the field of image processing, and relate to, but are not limited to, an image processing method, apparatus, device, and computer-readable storage medium.

Background

At present, the image processing is mainly characterized by coloring black-and-white photos, predicting image content, refining blurred photos and the like in image recognition such as face recognition, license plate recognition, character recognition and the like, and the processing method is mainly characterized by combining a traditional image recognition method based on rules with a processing method based on statistics. Post-processing of photography belongs to a relatively popular field in image processing, and is mainly aimed at processing original pictures by photographers or built-in software of cameras.

The photographic post-processing method in the related art mostly depends on image processing software (such as photoshop or lightroom software), for photographers, the processing of wind-solar photographic images is often a complicated and repeatable work, but good post-processing requires a certain sense of dexterity, which requires the photographers to spend a great deal of time learning various software operation steps, and for cameras, a good built-in post-processing software can also greatly increase the purchasing desire of customers.

However, the image processing in the related art is less applied in the aspect of post-processing of photography, and the traditional technical scheme is difficult to accurately reconstruct the image, so that the problems of lower accuracy and poorer processing effect of post-processing of the image in the post-processing process of the wind-solar photographic image exist in the image processing method in the related art.

Disclosure of Invention

The embodiment of the application provides an image processing method, an image processing device, image processing equipment and a computer readable storage medium, which can reduce time cost caused by learning various post-processing software, accurately identify the category of an image to be processed, accurately reconstruct the image to be processed according to the accurate category, obtain the processed image meeting the processing requirements of a user, and improve the accuracy and the processing effect of the post-processing of the image.

The technical scheme of the embodiment of the application is realized as follows:

In a first aspect, an embodiment of the present application provides an image processing method, including:

Extracting features of an image to be processed to obtain direction features and image features of the image to be processed;

identifying the direction characteristics to obtain the category of the image to be processed;

Performing image reconstruction processing on the image to be processed according to the category and the image characteristics to obtain a processed image;

And outputting the processed image.

An embodiment of the present application provides an image processing apparatus including:

The feature extraction module is used for extracting features of the image to be processed to obtain direction features and image features of the image to be processed;

the identification module is used for identifying the direction characteristics to obtain the category of the image to be processed;

The image reconstruction module is used for carrying out image reconstruction processing on the image to be processed according to the category and the image characteristics to obtain a processed image;

And the output module is used for outputting the processed image.

An embodiment of the present application provides an image processing apparatus including:

The image processing device comprises a memory for storing executable instructions, and a processor for realizing the image processing method when executing the executable instructions stored in the memory.

The embodiment of the application provides a storage medium, which stores executable instructions for realizing the image processing method when being executed by a processor.

According to the image processing method, the device, the equipment and the computer readable storage medium, the direction characteristics and the image characteristics of the image to be processed are extracted, and the image to be processed is processed based on the direction characteristics and the image characteristics, so that the type of the image to be processed can be accurately identified, the image to be processed is accurately reconstructed according to the accurate type, the accuracy and the processing effect of the post-processing of the image are improved, and the processed image meeting the processing requirements of a user is obtained.

Drawings

FIG. 1 is a schematic flow chart of an alternative image processing method according to an embodiment of the present application;

FIG. 2A is a schematic diagram of an image processing system according to an embodiment of the present application;

FIG. 2B is a schematic flow chart of an alternative image processing method according to an embodiment of the present application;

FIG. 3 is a schematic flow chart of an alternative image processing method according to an embodiment of the present application;

FIG. 4 is a schematic flow chart of an alternative image processing model training method according to an embodiment of the present application;

FIG. 5 is a schematic flow chart of an alternative image processing model training method according to an embodiment of the present application;

FIG. 6 is a schematic flow chart of an alternative image processing model training method according to an embodiment of the present application;

FIG. 7 is a schematic diagram of an implementation process of a model construction stage provided by an embodiment of the present application;

FIG. 8 is a schematic diagram of a first convolutional neural network model provided in an embodiment of the present application;

FIG. 9 is a schematic diagram of a second convolutional neural network model provided in an embodiment of the present application;

FIG. 10 is a schematic diagram of an implementation process of a model training phase provided by an embodiment of the present application;

FIG. 11 is a schematic flow chart of an image recognition stage according to an embodiment of the present application;

fig. 12 is a schematic diagram of a composition structure of an image processing apparatus provided in an embodiment of the present application;

Fig. 13 is a schematic diagram of a composition structure of an image processing apparatus provided in an embodiment of the present application.

Detailed Description

The present application will be further described in detail with reference to the accompanying drawings, for the purpose of making the objects, technical solutions and advantages of the present application more apparent, and the described embodiments should not be construed as limiting the present application, and all other embodiments obtained by those skilled in the art without making any inventive effort are within the scope of the present application.

In the following description, reference is made to "some embodiments" which describe a subset of all possible embodiments, but it is to be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments and can be combined with one another without conflict. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which embodiments of this application belong. The terminology used in the embodiments of the application is for the purpose of describing embodiments of the application only and is not intended to be limiting of the application.

In order to better understand the image processing method provided in the embodiment of the present application, first, an image processing method and problems existing in the related art will be described:

In the related art, the image processing is mostly performed on the recognition of the image, and the image processing mainly comprises coloring processing of black-and-white photos, image content prediction, refinement of blurred photos and the like, and the processing method mainly uses a combination of a traditional rule-based image recognition method and a statistics-based processing method.

The method in the related art has at least the following problems:

1) The traditional rule-based image recognition method is difficult to achieve a good recognition effect on the content classification of the image, and when the content complexity of the image reaches a certain degree, the recognition degree is linearly reduced. For example, for wind-light photography, the content is complex and various, and is more difficult to identify than traditional simple images such as faces, characters and the like.

2) In the related art, post-processing is rarely performed on the whole image, and most of the post-processing is performed on some part of the characteristic of the image or the content identification of the image. Such as sharpness processing of images, black and white photo coloring, face recognition, text recognition, are less involved in post-processing of photographic images.

3) Although a single convolutional neural network has good effects on the classification of certain pictures, the phenomenon of over fitting is easy to generate on a training set, the single convolutional neural network falls into local optimum, and the recognition rate on a verification set is often relatively low.

4) In the post-processing technology of camera software in the related art, although the software can perform corresponding post-processing according to corresponding scenes to directly obtain a photo, the method is still relatively single, for example, in a certain brand of mobile phone, the method can only perform content recognition according to a plurality of scenes such as blue sky, fresh flowers, sunset and the like, then perform post-processing, and the processing method is also simple in terms of adjustment of High dynamic range images (HDR, high-DYNAMIC RANGE), saturation, contrast, exposure and the like, so that the post-processing method of a general wind and light photographer is difficult to achieve.

5) Some image processing software such as lightroom, photoshop in the related art uses preset and filter modes, which can perform some automatic processing, but has stronger pertinence, and is only suitable for photographic works of the same style. For wind-light photographic works, the content forms are more complex and changeable, and the later requirements of photographers cannot be met by only using related techniques such as presetting.

6) In various industrialized photographic institutions, the post-photographic processing needs to take huge manpower, but the post-photographic processing is a work with high repeatability, most of working steps are almost the same, and how to quickly and well repair a large number of homogeneous photos is a problem to be solved.

Based on at least one of the above problems in the related art, in order to better integrate precious graphic repair resources, a photographer can make more creative work (such as more pre-shots) by freeing up his hands, so that a neural network is combined with post-processing of an image, and an image processing method is provided, which uses a convolutional neural network to perform post-processing of wind-solar photography. Extracting features of an image to be processed to obtain direction features and image features of the image to be processed, identifying the direction features to obtain categories of the image to be processed, carrying out image reconstruction processing on the image to be processed according to the categories and the image features to obtain a processed image, and outputting the processed image. Therefore, the type of the image to be processed can be accurately identified, so that accurate image reconstruction processing is carried out on the image to be processed according to the accurate type, and the processed image meeting the processing requirements of the user is obtained.

An exemplary application of the image processing apparatus provided by the embodiment of the present application is described below, and the image processing apparatus provided by the embodiment of the present application may be implemented as a notebook computer, a tablet computer, a desktop computer, a mobile device (for example, a mobile phone, a portable music player, a personal digital assistant, a dedicated messaging device, a portable game device), and other various types of terminals, and may also be implemented as a server. In the following, an exemplary application when the program running device is implemented as a server will be described.

Referring to fig. 1, fig. 1 is a schematic flowchart of an alternative image processing method according to an embodiment of the present application, and will be described with reference to the steps shown in fig. 1.

Step S101, extracting features of an image to be processed to obtain direction features and image features of the image to be processed.

Here, the image to be processed may be a picture downloaded from a website, or may be a picture taken by a user through an image pickup apparatus, for example, may be a wind-light photograph taken by the user.

In the embodiment of the application, when the feature extraction is carried out, different feature extraction tools can be respectively adopted to carry out the extraction of the direction feature and the extraction of the image feature of the image to be processed. The direction feature is a feature representing the directivity of the image, and the extraction of the direction feature is to actually identify the features of the image in different directions, so that the selectivity of human eyes to the direction in image identification is simulated. In the embodiment of the present application, the direction feature is mainly extracted to obtain some local features of the image, that is, the direction feature extracted in the embodiment of the present application is a local feature on the image to be processed. Image features refer to features that characterize the association between multiple image pixels, and may be global features.

And step S102, identifying the direction characteristics to obtain the category of the image to be processed.

Here, the category of the image to be processed is determined according to the direction characteristics, and the category may be any image category, for example, the category may be a landscape category such as a mountain, a water flow, a building, a night scene, a sunset, a sky, a plant, a road, a snow scene, a fresh flower, a magnificent, a mystery, negligible, a beach, or a person category such as an expression, a face, or the like.

And step S103, carrying out image reconstruction processing on the image to be processed according to the category and the image characteristics to obtain a processed image.

Here, the image post-processing of the image to be processed is implemented through the image reconstruction processing, where different image types correspond to different processing modes, for example, for a wind-light photographing picture, the content included in the picture is complex and the information is more, and for a person picture, the content included in the picture is simple and the information is less, so that different processing methods are required, and different requirements are also required for the overall effects of the wind-light photographing picture and the person picture on the picture. Therefore, the image to be processed can be subjected to later image reconstruction processing according to the category of the image to be processed, so that the processed image meeting the requirement of the picture is obtained.

Step S104, outputting the processed image.

According to the image processing method provided by the embodiment of the application, the direction characteristics and the image characteristics of the image to be processed are extracted, and the image to be processed is processed based on the direction characteristics and the image characteristics, so that the type of the image to be processed can be accurately identified, the image to be processed is accurately reconstructed according to the accurate type, the accuracy and the processing effect of the post-processing of the image are improved, and the processed image meeting the processing requirements of a user is obtained.

The image processing method according to the embodiment of the present application may be applied to post-processing of wind-solar photography, where the image processing method according to the embodiment of the present application may be applied to an image processing system, and fig. 2A is a schematic structural diagram of the image processing system according to the embodiment of the present application, and as shown in fig. 2A, the image processing system 20 includes a terminal 100, a network 200, and a server 300. After a photographer shoots a wind-solar shooting image, the image is input as an image to be processed to the terminal 100, an image processing Application (APP) is installed on the terminal 100, and the terminal 100 sends the image to be processed to a server 300 of the image processing application through a network 200, so that the server 300 performs image post-processing on the image to be processed to obtain a processed image. At least one of the image to be processed and the processed image may be displayed on the current interface 100-1 of the terminal 100.

Fig. 2B is a schematic flowchart of an alternative image processing method according to an embodiment of the present application, as shown in fig. 2B, based on the image processing system of fig. 2A, the image processing method includes the following steps:

in step S201, the user inputs the wind-solar photographic image as an image to be processed to the terminal.

In step S202, the terminal sends the image to be processed to the server.

Step S203, the server performs feature extraction on the image to be processed to obtain local features and overall features of the image to be processed.

Here, the local feature corresponds to the above-described directional feature, and the global feature corresponds to the above-described image feature.

Step S204, the server identifies the local features and the whole features to obtain the category of the image to be processed.

In some embodiments, in determining the class of the image to be processed, this may be achieved by:

Step S2041, carrying out convolution processing and pooling processing on the local features in sequence to obtain pooled processing features.

Here, the local feature is a local feature of a local region in the image to be processed, and because the image features of not all regions in the whole image to be processed are valid features and cannot be used as the basis for determining the image category, the image category identification is performed by acquiring the local feature of the image to be processed.

In the embodiment of the present application, the local feature may be a feature of a region with more feature data in the image to be processed, or a feature of a region with a larger change in feature data, for example, if an exposure region exists in the image to be processed, the feature of the region cannot be used as an effective feature to perform image classification recognition, or if a region with more objects (such as a person, a landscape, and a building) exists in the image to be processed, the region is the region capable of determining the image classification most accurately, and the change between the feature data of the region is larger (i.e., the difference between the pixel values of the pixel points is larger), so that the feature of the region is determined as the local feature.

In the embodiment of the application, the local features can be subjected to convolution processing through a convolution layer in the convolution neural network, and the local features after the convolution processing are subjected to pooling processing through a pooling layer in the convolution neural network.

Step S2042, performing recognition processing on the pooled processing features to determine a class corresponding to the pooled processing features.

Here, by comparing the pooling feature with preset features of different categories preset in the convolutional neural network, a feature difference between the pooling feature and each preset feature is determined, the preset feature with the smallest feature difference is determined as a similar feature of the pooling feature, and finally, the category of the similar feature is determined as the category of the pooling feature.

And step S2043, determining the category corresponding to the pooling processing characteristic as the category of the image to be processed.

In step S205, the server performs image reconstruction processing on the image to be processed according to the category and the overall characteristics, and obtains a processed image.

In step S206, the server transmits the processed image to the terminal.

In step S207, the terminal displays the processed image on the current interface.

According to the image processing method provided by the embodiment of the application, the terminal sends the image to be processed to the server for processing, and the server extracts the local characteristics and the whole characteristics of the image to be processed and processes the image to be processed based on the local characteristics and the whole characteristics, so that the type of the image to be processed can be accurately identified according to the local characteristics and the whole characteristics, the image to be processed is accurately reconstructed according to the accurate type, the processed image meeting the user processing requirement is obtained, and the processed image meeting the user processing requirement is displayed on the terminal.

In some embodiments, the image processing method of embodiments of the present application may be implemented in conjunction with a filter and convolutional neural network. Based on fig. 1 and fig. 3 are schematic flow diagrams of an alternative image processing method according to an embodiment of the present application, as shown in fig. 3, step S101 may be implemented by:

Step S301, performing fourier transform processing on the image to be processed by using a filter, to obtain a direction feature of the image to be processed.

Here, the filter may be a Gabor filter, and in the embodiment of the present application, the Gabor filter may be used to perform fourier transform processing on the image to be processed to obtain the directional characteristic of the image to be processed. Wherein the Gabor filter is obtained by superimposing a gaussian function with a trigonometric function (for example, a sine function). The pixels of the image to be processed may be subjected to a transformation process of a trigonometric function and a gaussian function by a Gabor filter, and the values of the transformation process may be determined as the direction characteristics.

And step S302, carrying out feature extraction on the image to be processed by adopting a convolutional neural network to obtain the image features of the image to be processed.

The method comprises the steps of carrying out feature extraction on an image to be processed by adopting a convolutional neural network, wherein the convolutional neural network for realizing feature extraction at least comprises a convolutional layer and a pooling layer, carrying out convolutional processing on the image to be processed by a certain number of the convolutional layers, and carrying out pooling processing on the image to be processed by a certain number of the pooling layers, so that the image features of the image to be processed are obtained, and the image features are basic features for carrying out image processing.

With continued reference to fig. 3, step S103 may be implemented by:

and step S303, determining the convolutional neural network corresponding to the category as a target convolutional neural network.

And step S304, performing image reconstruction processing on the image features by adopting the target convolutional neural network to obtain the processed image.

In the embodiment of the application, after the category of the image to be processed is determined, a target convolutional neural network is acquired according to the category of the image to be processed, and the acquired target convolutional neural network is adopted for image reconstruction processing.

In some embodiments, whether the image to be processed is to be processed may also be determined according to a convolutional neural network, that is, the type corresponding to the convolutional neural network is fixed, the convolutional neural network is used for performing image reconstruction processing on the image to be processed of a specific type, and if the type of the image to be processed is of the specific type, the convolutional neural network is used for processing the image to be processed. The specific category is a preset category, the preset category can be a category preset by a user, and the preset category can be set according to the actual processing requirement of the user or according to the image category which can be processed by software for realizing image processing. For example, the specific category may be a wind-light photograph picture category, or the specific category may also be a mountain category in a wind-light photograph picture, or the like.

In some embodiments, the image processing method according to the embodiments of the present application may also be implemented using an image processing model, that is, an image processing model is used to perform image reconstruction processing on the image to be processed. The image processing model comprises a filter, an image category recognition sub-model and an image reconstruction sub-model, wherein the filter is used for extracting characteristics of an image to obtain direction characteristics of the image, the image category recognition sub-model is used for recognizing the type of the image, and the image reconstruction sub-model is used for recognizing the content of the image and carrying out image reconstruction processing on the image. The method mainly aims to acquire some local features of the image, and finally aims to extract the whole features of the image by combining the image type recognition sub-model and finally recognize the image type by combining the local features and the whole features.

Here, a training method of an image processing model is provided, as shown in fig. 4, which is an optional flowchart of the training method of an image processing model provided in the embodiment of the present application, where the method includes:

step S401, inputting a sample image into a filter, and obtaining the direction characteristic of the sample image.

Here, the sample image may be a picture downloaded by the user on the website, or may be a picture taken by the user using the photographing apparatus.

The filter is used for carrying out Fourier transform processing on the sample image so as to obtain the direction characteristics of the sample image. The extraction of the directional features of the sample image is to actually identify the features of the sample image in different directions, and the extraction is to simulate the selectivity of human eyes to directions in image identification. In the embodiment of the present application, the direction feature of the sample image is mainly used to obtain some local features of the sample image, that is, the direction feature of the sample image extracted in the embodiment of the present application is a local feature on the sample image.

Step S402, inputting the direction features into an image category recognition sub-model, and performing image category recognition processing on the direction features through the image category recognition sub-model to obtain the category of the sample image.

Here, the image category recognition sub-model is used to recognize the category of the sample image based on the direction feature. In the implementation process, convolution processing and pooling processing can be performed on the directional characteristics, and finally the category of the sample image is obtained.

Step S403, acquiring an image reconstruction sub-model corresponding to the category.

Step S404, inputting the sample image into an image reconstruction sub-model, and performing image reconstruction processing on the sample image through the image reconstruction sub-model to obtain a processed sample image.

The image reconstruction sub-model is used for extracting features of the sample image to obtain image features, and performing image reconstruction processing on the image features to obtain an image after image reconstruction, namely a processed sample image.

Step S405, inputting the category of the sample image and the processed sample image into a preset loss model, so as to obtain a loss result.

In some embodiments, the predetermined loss model comprises a first predetermined loss model and a second predetermined loss model, and the loss result comprises a first loss result and a second loss result. The first preset loss model is used for verifying the output result of the image category identification sub-model to obtain a first loss result, and the second preset loss model is used for verifying the output result of the image reconstruction sub-model to obtain a second loss result. Correspondingly, step S405 may be implemented by the following steps:

step S4051, inputting the category of the sample image into the first preset loss model, to obtain the first loss result.

The first preset loss model is used for comparing the category of the sample image with a preset category to obtain a first loss result, wherein the preset category can be obtained after the user manually identifies the category of the sample image.

The first preset loss model comprises a first loss function, a first similarity between the category of the sample image and the preset category can be calculated through the first loss function, and the first loss result is determined according to the first similarity.

Step S4052, inputting the processed sample image into the second preset loss model to obtain the second loss result.

The second preset loss model is used for comparing the processed sample image with a preset processing image to obtain a second loss result, wherein the preset processing image can be an image obtained by manually processing the sample image by a user.

The second preset loss model comprises a second loss function, a second similarity between the sample image and the preset processing image can be calculated through the second loss function, and the second loss result is determined according to the second similarity.

And step S406, correcting the image type recognition sub-model and the image reconstruction sub-model according to the loss result until the image type recognition sub-model and the image reconstruction sub-model can accurately process the sample image.

Here, when the above-mentioned similarity is greater than the preset similarity threshold, the loss result indicates that the image type recognition sub-model in the current image processing model cannot accurately recognize the type of the sample image, or indicates that the image reconstruction sub-model in the current image processing model cannot accurately process the sample image. Therefore, correction of the image class identification sub-model or the image reconstruction sub-model in the current image processing model is required. And then, correcting the image reconstruction sub-model according to the similarity until the similarity between the processed sample image output by the image processing model and the preset processing image meets the preset condition, and determining the corresponding image processing model as a trained image processing model.

Of course, in some embodiments, the image reconstruction sub-model in step S403 to step S406 may be any other model corresponding to a category, for example, a model corresponding to a category of mountain, or a model corresponding to a category of big tree, which is not limited in the embodiments of the present application.

According to the training method of the image processing model, the sample image is input into the filter to obtain the direction characteristic of the sample image, the direction characteristic is input into the image type recognition sub-model to obtain the type of the sample image, the type of the sample image is input into the first preset loss model to obtain the first loss result, the sample image is input into the image reconstruction sub-model to obtain the processed sample image, and the processed sample image is input into the second preset loss model to obtain the second loss result. Therefore, the image type recognition sub-model and the image reconstruction sub-model can be corrected according to the first loss result and the second loss result respectively, the obtained corrected image type recognition sub-model can accurately recognize the type of the image, the obtained corrected image reconstruction sub-model can accurately perform image post-processing on the image, and therefore the processed image meeting the user processing requirements is obtained, and the user experience is improved.

In the embodiment of the application, the image category recognition sub-model and the image reconstruction sub-model can be two independent training processes, the direction characteristics generated by the Gabor filter are input into the image category recognition sub-model, after a plurality of iterations, the image category recognition sub-model is compared with the pre-marked image category, and then the image category recognition sub-model is corrected according to the first loss result, the image reconstruction sub-model training process is that a sample image with known category information is input into the image reconstruction sub-model, and after a plurality of iterations, the second loss result is evaluated through a loss function, and model parameters are continuously corrected. Therefore, the image reconstruction sub-model of a plurality of categories (such as mountain, sunrise and the like) can be trained corresponding to the images of the plurality of categories, and in the image category identification process, the image category identified by the image category identification sub-model is only required to be input into the corresponding image reconstruction sub-model (for example, the image category identification sub-model identifies the mountain category, then the image can be input into the image reconstruction sub-model of the mountain category), so that the processing of the corresponding category can be performed.

In some embodiments, the image reconstruction sub-model not only can extract the image features of the sample image, but also can reconstruct the image of the sample image, based on fig. 4, as shown in fig. 5, is an optional flowchart of the image processing model training method provided by the embodiment of the present application, and step S404 may also be implemented by:

Step S501, determining an image reconstruction sub-model corresponding to the category of the sample image, inputting the sample image into the image reconstruction sub-model, and extracting features of the sample image through a feature extraction layer in the image reconstruction sub-model to obtain image features of the sample image.

Step S502, performing image reconstruction processing on the image features through an image reconstruction processing layer in the image reconstruction sub-model, to obtain the processed sample image.

Here, the image reconstruction processing layer includes a convolution layer and an upsampling layer, and correspondingly, the step S502 may be implemented by:

And step S5021, carrying out convolution processing on the image features through the convolution layer to obtain convolution processing features.

And step S5022, up-sampling the convolution processing characteristics through the up-sampling layer to obtain up-sampling characteristics.

Step S5023, determining the processed sample image according to the upsampling feature.

Based on fig. 4, fig. 6 is a schematic flow chart of an optional image processing model training method according to an embodiment of the present application, as shown in fig. 6, where the method further includes:

Step S601, inputting a preset category corresponding to the sample image into a first preset loss model, and inputting a processed image corresponding to the sample image into a second preset loss model.

It should be noted that, step S601 may be performed at any time before step S405, which is not limited in the embodiment of the present application.

Correspondingly, step S405 may be implemented by the following steps:

Step S602, determining, by using the first preset loss model, a first similarity between the category of the sample image and the preset category.

Step S603, determining the first loss result according to the first similarity.

Step S604, determining a second similarity between the processed sample image and the processed image through the second preset loss model.

Step S605, determining the second loss result according to the second similarity.

Correspondingly, step S406 may be implemented by:

and step S606, correcting the image category recognition sub-model according to the first loss result until the image category recognition sub-model can accurately determine the category of the sample image.

Step S607, correcting the image reconstruction sub-model according to the second loss result until the image reconstruction sub-model can accurately perform image processing on the sample image.

In the following, an exemplary application of the embodiment of the present application in a practical application scenario will be described.

The embodiment of the application provides an image processing method, which utilizes a convolutional neural network to carry out post-processing of wind-light photography and mainly uses the convolutional neural network and a Gabor filter. The convolutional neural network is a deep feed-forward neural network with a plurality of convolutional layers and pooling layers, the local features of the image are extracted through the convolutional layers and are input into an activation function, and after normalization processing is carried out, the classification result of the image is output. The convolutional neural network mainly comprises a training stage and a classifying stage, a large number of sample sets are needed in the training stage, the recognition results of the large number of sample sets are compared with labels agreed in advance, and the labels are conducted reversely, so that the weight in the network is updated, and after a large number of iterations update the weight, the optimal solution is obtained, so that the purpose of recognizing the image is achieved. The Gabor filter can extract features of the image in different directions by using fourier transform.

The method mainly comprises a model construction stage, a model training stage and an image recognition stage.

Fig. 7 is a schematic diagram of an implementation process of a model construction stage according to an embodiment of the present application, as shown in fig. 7, including the following steps:

In step S701, image classification is performed.

Here, the overall characteristics of the image may be identified and the subject matter of the image classified by the image dataset (e.g., imageNet dataset) and the user-collected image collection together. For example, in the present application, 15 different classifications of wind-light images can be made, mainly including emotion and scenery, including mountain, water flow, building, night scene, sunset, starry sky, plant, road, snow scene, flower, ambitious, mystery, negligible, beach, quiet, etc., and these labels can cover most of the subjects in wind-light photography. This classification work is done prior to image training, with the image types being manually labeled.

Step S702, a Gabor filter is constructed.

The Gabor filter is added in the convolutional neural network model, so that the convolutional neural network can be added to identify the image direction, and the identification precision is increased. The Gabor filter is a window function that introduces a local change in time from the fourier transform, resulting in a windowed fourier transform. The real form of the Gabor filter is shown in the following formula (1-1):

wherein x '= xcos θ+ ysin θ, y' = -xsin θ+ ycos θ, and x and y represent coordinate values of the Gabor filter on the abscissa and the ordinate, respectively.

Λ represents wavelength, θ represents direction, the value range is 0 to 360 degrees, ψ represents phase offset, the value range is-180 degrees to 180 degrees, σ represents standard deviation of Gabor function Gaussian factor, γ represents space length-width ratio, when γ=1, shape is round, when γ <1, shape is elongated along with parallel stripe direction, and the value can be 0.5.

In order to extract Gabor features (namely direction features) in 8 directions of 0, pi/8, pi/4, 3 pi/8, pi/2, 5 pi/8, 3 pi/4 and 7 pi/8, the embodiment of the application constructs Gabor filters with 4 different scales, each scale being different by 90 degrees and the phase difference being 8 different directions.

In step S703, a first convolutional neural network model (corresponding to the above-described image class recognition sub-model) is constructed.

The model classified here uses a convolutional neural network model that contains three convolutional layers, three pooled layers, and one normalized layer (softmax layer). The three convolution layers and the three pooling layers mainly identify local features of the image, and the softmax layer contains 15 neurons and is used for classifying subjects of the image subjected to Gabor feature extraction and local feature extraction.

As shown in fig. 8, a schematic structural diagram of a first convolutional neural network model provided in an embodiment of the present application, the first convolutional neural network model 80 includes an input layer 801, a first convolutional layer 802, a first pooling layer 803, a second convolutional layer 804, a second pooling layer 805, a third convolutional layer 806, a third pooling layer 807, a first full connection layer 808, a hyperbolic tangent function layer (Tanh layer) 809, a second full connection layer 810, and a softmax layer 811, which are sequentially connected.

Wherein, the input layer 801 inputs the features h×w, and the first convolution layer 802 convolves the n features h×w to obtain 2n features h×w; the method comprises the steps of carrying out pooling treatment on 2n characteristics H/2 xW/2 through a first pooling layer 803 to obtain 2n characteristics H/2 xW/2, carrying out convolution treatment on 2n characteristics H/2 xW/2 through a second convolution layer 804 to obtain 4n characteristics H/2 xW/2, carrying out pooling treatment on 4n characteristics H/2 xW/2 through a second pooling layer 805 to obtain 4n characteristics H/4 xW/4, carrying out convolution treatment on 4n characteristics H/4 xW/4 through a third convolution layer 806 to obtain 8n characteristics H/4 xW/4, carrying out pooling treatment on 8n characteristics H/4 xW/4 through a third pooling layer 807 to obtain 8n characteristics H/8 xW/8, carrying out full connection treatment on 8n characteristics H/8 xW/8 through a first full connection layer 808 to obtain 8n characteristics 1 x1, carrying out tangent normalization treatment on the 4n characteristics H/4 xW/4 through a Tanh layer 809, a second full connection layer and a sotmax layer 810 in sequence, carrying out tangent type normalization treatment on the characteristics 1 xW/8, carrying out full connection treatment on the characteristics 1 xW/4 in sequence, and carrying out full connection type calculation on the characteristics 1 xW/8, and carrying out positive type normalization treatment in sequence, and determining the final type treatment. n is a positive integer, for example, n may take on a value of 32.

In step S704, a second convolutional neural network model (corresponding to the image reconstruction sub-model described above) is constructed.

The second convolutional neural network is mainly divided into two stages, namely a feature extraction stage and an image reconstruction stage. The feature extraction stage comprises three convolution layers, and the image reconstruction stage comprises three convolution layers and three up-sampling layers.

As shown in fig. 9, a schematic structural diagram of a second convolutional neural network model provided in an embodiment of the present application, where the second convolutional neural network model 90 includes a feature extraction stage 91 and an image reconstruction stage 92, and includes an input layer 901, a first convolutional layer 902, a second convolutional layer 903, a first pooling layer 904, a third convolutional layer 905, a second pooling layer 906, a convolutional pooling layer 907, a first upsampling layer 908, a second upsampling layer 909, a fourth convolutional layer 910, a third upsampling layer 911, a fifth convolutional layer 912, a sixth convolutional layer 913, and an output layer 914 that are sequentially connected.

In the feature extraction stage, the input layer 901 inputs m features h×w, the first convolution layer 902 convolves the m features h×w to obtain 2m features h×w, the second convolution layer 903 convolves the 2m features h×w to obtain 4m features h×w, the first pooling layer 904 pools the 4m features h×w to obtain 4m features H/2×w/2, the third convolution layer 905 convolves the 4m features H/2×w/2 to obtain 8m features H/2×w/2, the second pooling layer 906 pools the 8m features H/2×w/2 to obtain 8m features H/4×w/4, and the convolution pooling layer 907 convolves the 8m features H/4×w/4 to obtain 16m features H/8×w/8, thereby completing the feature extraction process.

In the image reconstruction stage, the 16m features H/8×w/8 extracted in the feature extraction stage are up-sampled by the first up-sampling layer 908 to obtain 8m features H/4×w/4, the 8m features H/4×w/4 are up-sampled by the second up-sampling layer 909 to obtain 8m features H/2×w/2, the 8m features H/2×w/2 are convolved by the fourth convolution layer 910 to obtain 4m features H/2×w/2, the 4m features H/2×w/2 are up-sampled by the third up-sampling layer 911 to obtain 4m features h×w, the 4m features h×w are convolved by the fifth convolution layer 912 to obtain 2m features h×w, the 2m features h×w are convolved by the sixth convolution layer 913, and finally the m features h×w are output by the output layer 914. m is a positive integer, for example, m may take on a value of 32.

Based on the above model construction stage, fig. 10 is a schematic diagram of an implementation process of the model training stage according to the embodiment of the present application, as shown in fig. 10, including the following steps:

in step S1001, the sample image is input to the Gabor filter for extracting the direction feature, and the direction feature is input to the first convolutional neural network model for performing the category recognition.

Here, one picture hxw×3 having three channels of RGB may be input to the Gabor filter, and convolved with 4×8×3 filters to obtain a graphic feature (i.e., a directional feature) having a thickness of 8 channels having hxw×8.

The method comprises the steps of inputting the direction characteristics into a content recognition convolutional neural network model (namely a first convolutional neural network model), firstly convolving with a convolutional kernel with the size of H multiplied by 3, obtaining an image with the size of H multiplied by W multiplied by H, then carrying out maximum sampling (pooling) on the image, wherein the sliding step is 2, the image size is reduced to 1/2 of the original size, obtaining an image with the size of H/2 multiplied by W/2 multiplied by H, carrying out convolution and pooling operations twice according to the steps in sequence, finally outputting the image to a softmax layer, obtaining 15 graphic characteristics with the size of 1 multiplied by 1, carrying out normalization processing by using a softmax function as an activation function, and finally obtaining the type of the image.

Step S1002, inputting the sample image into a second convolutional neural network model for image post-processing.

Here, the image post-processing is performed using the second convolutional neural network model, and a raw sheet of RGB channels (i.e., a sample image) that has not undergone the post-processing is input into the second convolutional neural network model, which corresponds to inputting a graph of hxw×3 having a 3-channel thickness into the network model. All that is required for convolutional neural networks is to predict the distribution of the values of each pixel point in the RGB three channels from 0 to 255 according to a certain parametric model. The image processing process mainly comprises two stages, namely a feature extraction stage and an image reconstruction stage. The method comprises the steps of firstly convolving an image with H multiplied by W multiplied by 3 with convolution kernels with the sizes of 3 multiplied by 64 and 3 multiplied by 128 to obtain an image with the size of H multiplied by 128, then convolving the image with a convolution kernel with the size of 3 multiplied by 3 and with a sliding step length of 2 to obtain H/2 multiplied by W/2 multiplied by 128, and carrying out three convolution and pooling operations to obtain an image with the size of H/8 multiplied by W/8 multiplied by 512, which is a characteristic extraction process, wherein the image reconstruction process uses a convolution kernel with the size of 3 multiplied by 3, and the sliding step length is 1, and the image with the size of H multiplied by W multiplied by 3 is obtained again after the convolution and the up sampling for many times.

Step S1003, optimizing the first convolutional neural network model and the second convolutional neural network model.

In the image training stage, the reconstructed image is compared with a target image (which can be an image processed by human later stage), the weight in the network is updated through back propagation, and the optimal parameter model is obtained after a plurality of iterations, namely the training of the second convolutional neural network is completed.

It should be noted that, the training process of the first convolutional neural network model continuously corrects the parameter model by predicting the label category of the training image, and the training process of the second convolutional neural network model corrects the parameter model by comparing the content of the training image and the content of the target image. The training process of the two models is two relatively independent processes.

In the image recognition stage, the image recognition may be performed using the model constructed and trained in the above embodiment. In the image recognition process, an image to be recognized firstly passes through a first convolution neural network, after the recognition of an image subject is completed, the image is input into a second convolution neural network model, the image is subjected to post-processing, and finally a picture subjected to certain post-processing is obtained.

Fig. 11 is a schematic flow chart of an image recognition stage according to an embodiment of the present application, as shown in fig. 11, including the following steps:

in step S1101, an image training set is formed from images crawled from ImageNet and each large photographing website, and the image training set is input to a Gabor filter to obtain a direction feature.

Step S1102, inputting the direction feature into the first convolutional neural network model to obtain the category of the image.

Step S1103, verifying the category of the image according to the verification set for the category of the image, so as to obtain a parameter model of the first convolutional neural network model.

Step S1104, obtaining an image classification according to the parameter model.

Step S1105, forming a picture set from the images crawled from each large photographing website, and inputting the picture set into the second convolutional neural network model to obtain the processed image features.

And step S1106, processing the processed image features according to the acquired image classification and the corresponding type parameter model to obtain a post-processing result.

Step S1107, outputting the post-processing result.

The image processing method provided by the embodiment of the application combines the convolutional neural network with the Gabor filter to identify, classify and post-process the image.

The convolutional neural network and the Gabor filter are combined, so that the content of the image can be better identified, and the direction characteristic of the image can be identified mainly due to the fact that the Gabor filter is introduced, the fact that the convolutional neural network can only identify the local characteristic of the image is made up, and the method has a certain similarity with the identification principle of human eyes.

Since the post-processing of the photographic image is rarely performed or the processing method is relatively single in the image processing technology in the related art, the post-processing of the photographic image is performed through the convolutional neural network, which is more advantageous than the rule-based recognition method in the related art.

Compared with the photographic post-processing technology manually operated in the related art, the method of the embodiment of the application can reduce the time cost brought by people in learning various post-processing software, achieves the aim of automatically post-processing pictures by training a large number of picture data sets through adopting the convolutional neural network, can liberate the productivity to a certain extent, and can realize one-key processing of the images to be processed because the convolutional neural network and the Gabor filter are integrated in the same image processing software.

Based on the foregoing embodiments, the embodiments of the present application provide an image Processing apparatus, where the apparatus includes each module included, and each component included in each module may be implemented by a processor in an image Processing device, or may of course be implemented by a logic circuit, and in the implementation process, the processor may be a central Processing unit (CPU, central Processing Unit), a microprocessor (MPU, micro Processor Unit), a digital signal processor (DSP, digital Signal Processing), or a field programmable gate array (FPGA, field Programmable GATE ARRAY), etc.

Fig. 12 is a schematic diagram of the composition structure of an image processing apparatus according to an embodiment of the present application, and as shown in fig. 12, the image processing apparatus 1200 includes:

the feature extraction module 1201 is configured to perform feature extraction on an image to be processed to obtain a direction feature and an image feature of the image to be processed;

The identifying module 1202 is configured to identify the direction feature, so as to obtain a category of the image to be processed;

the image reconstruction module 1203 is configured to perform image reconstruction processing on the image to be processed according to the category and the image feature, to obtain a processed image;

and the output module 1204 is used for outputting the processed image.

In some embodiments, the direction feature is a local feature of a local area in the image to be processed, and the identification module is further configured to sequentially perform convolution processing and pooling processing on the local feature to obtain pooled processing features, perform identification processing on the pooled processing features to determine a class corresponding to the pooled processing features, and determine the class corresponding to the pooled processing features as the class of the image to be processed.

In some embodiments, the feature extraction module is further configured to perform fourier transform processing on the image to be processed by using a filter to obtain a directional feature of the image to be processed, and perform feature extraction on the image to be processed by using a convolutional neural network to obtain an image feature of the image to be processed.

In some embodiments, the image reconstruction module is further configured to determine that the convolutional neural network corresponding to the category is a target convolutional neural network, and perform image reconstruction processing on the image feature by using the target convolutional neural network to obtain the processed image.

In some embodiments, the device further comprises a control module, wherein the control module is used for carrying out image reconstruction processing on the image to be processed by adopting an image processing model, the image processing model is obtained through training, a sample image is input into the image processing model to be trained, the sample image is subjected to image processing by the image processing model to be trained, the category of the sample image and the processed sample image are obtained, the category of the sample image and the processed sample image are input into a preset loss model, a loss result is obtained, and parameters in the image processing model to be trained are corrected according to the loss result, so that a corrected image processing model is obtained.

In some embodiments, the image processing model to be trained comprises a filter and an image class identification sub-model, wherein the image processing model is trained by performing Fourier transform processing on the sample image through the filter to obtain the direction characteristic of the sample image, and performing image class identification processing on the direction characteristic through the image class identification sub-model to obtain the class of the sample image.

In some embodiments, the image processing model to be trained further comprises an image reconstruction sub-model corresponding to the category of the sample image, and correspondingly, the image processing model is obtained through training through the steps that the image reconstruction sub-model is used for carrying out image reconstruction processing on the sample image, and the processed sample image is obtained.

In some embodiments, the image reconstruction sub-model comprises a feature extraction layer and an image reconstruction processing layer, wherein the image processing model is trained by performing feature extraction on the sample image through the feature extraction layer to obtain image features of the sample image, and performing image reconstruction processing on the image features through the image reconstruction processing layer to obtain the processed sample image.

In some embodiments, the image reconstruction processing layer comprises a convolution layer and an up-sampling layer, the image processing model is trained by performing convolution processing on the image features through the convolution layer to obtain convolution processing features, performing up-sampling processing on the convolution processing features through the up-sampling layer to obtain up-sampling features, and determining the processed sample image according to the up-sampling features.

In some embodiments, the apparatus further comprises an input module for inputting a preset category corresponding to the sample image and a processed image corresponding to the sample image into the preset loss model;

correspondingly, the image processing model is obtained through training by determining a first similarity between the category of the sample image and the preset category, determining a second similarity between the processed sample image and the processed image, and determining the loss result according to the first similarity and the second similarity.

It should be noted that, the description of the apparatus according to the embodiment of the present application is similar to the description of the embodiment of the method described above, and has similar beneficial effects as the embodiment of the method, so that a detailed description is omitted. For technical details not disclosed in the present apparatus embodiment, please refer to the description of the method embodiment of the present application for understanding.

It should be noted that, in the embodiment of the present application, if the above-mentioned image processing method is implemented in the form of a software functional module, and sold or used as a separate product, it may also be stored in a computer readable storage medium. Based on such understanding, the technical solution of the embodiments of the present application may be essentially or part contributing to the related art, embodied in the form of a software product stored in a storage medium, including several instructions for causing a terminal to execute all or part of the methods described in the embodiments of the present application. The storage medium includes various media capable of storing program codes, such as a usb (universal serial bus), a removable hard disk, a Read Only Memory (ROM), a magnetic disk, or an optical disk. Thus, embodiments of the application are not limited to any specific combination of hardware and software.

Correspondingly, an embodiment of the present application provides an image processing apparatus, fig. 13 is a schematic diagram of a composition structure of the image processing apparatus provided by the embodiment of the present application, and as shown in fig. 13, the image processing apparatus 1300 includes at least a processor 1301, a communication interface 1302, and a computer readable storage medium 1303 configured to store executable instructions, where the processor 1301 generally controls an overall operation of the image processing apparatus. The communication interface 1302 may enable the image processing apparatus to communicate with other terminals or servers through a network. The computer-readable storage medium 1303 is configured to store instructions and applications executable by the processor 1301, and may also cache data to be processed or processed by each module in the processor 1301 and the image processing apparatus 1300, and may be implemented by a FLASH memory (FLASH) or a random access memory (RAM, random Access Memory).

It should be appreciated that reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. It should be understood that, in various embodiments of the present application, the sequence numbers of the foregoing processes do not mean the order of execution, and the order of execution of the processes should be determined by the functions and internal logic thereof, and should not constitute any limitation on the implementation process of the embodiments of the present application. The foregoing embodiment numbers of the present application are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.

Without further limitation, an element defined by the phrase "comprising one does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element. In the several embodiments provided by the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is merely a logical function division, and there may be additional divisions of actual implementation, such as multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed.

The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, may be located in one place or distributed on a plurality of network units, and may select some or all of the units according to actual needs to achieve the purposes of the embodiment of the present application. It will be appreciated by those of ordinary skill in the art that implementing all or part of the steps of the above method embodiments may be accomplished by hardware associated with program instructions, and that the above program may be stored on a computer readable storage medium which, when executed, performs the steps comprising the above method embodiments, where the above storage medium includes a removable storage device, a read only memory, a magnetic or optical disk, or other various media capable of storing program code. Or the above-described integrated units of the application may be stored in a computer-readable storage medium if implemented in the form of software functional modules and sold or used as separate products. Based on such understanding, the technical solution of the embodiments of the present application may be essentially or part contributing to the related art, embodied in the form of a software product stored in a storage medium, including several instructions for causing a terminal to execute all or part of the methods described in the embodiments of the present application. The storage medium includes various media capable of storing program codes such as a removable storage device, a ROM, a magnetic disk, or an optical disk.

Claims (11)

1. An image processing method, comprising:

carrying out Fourier transform processing on an image to be processed by adopting a filter to obtain direction characteristics of the image to be processed, wherein the direction characteristics are local characteristics of local areas in the image to be processed;

Performing feature extraction on the image to be processed by adopting a convolutional neural network to obtain image features of the image to be processed;

The local features are subjected to convolution processing and pooling processing in sequence to obtain pooling processing features, the pooling processing features are subjected to identification processing to determine the category corresponding to the pooling processing features, and the category corresponding to the pooling processing features is determined as the category of the image to be processed;

According to the type of the image to be processed and the image characteristics, performing image reconstruction processing on the image to be processed by adopting an image processing model to obtain a processed image, wherein the image processing model comprises a filter, an image type recognition sub-model and an image reconstruction sub-model corresponding to the type, and the image type recognition sub-model is used for recognizing the type of the image to be processed;

And outputting the processed image.

2. The method according to claim 1, wherein the method further comprises:

determining the convolutional neural network corresponding to the category as a target convolutional neural network;

And carrying out image reconstruction processing on the image features by adopting the target convolutional neural network to obtain the processed image.

3. The method according to claim 1, wherein the training method of the image processing model comprises the steps of:

Inputting a sample image into an image processing model to be trained;

performing image processing on the sample image through the image processing model to be trained to obtain the category of the sample image and the processed sample image;

inputting the category of the sample image into a first preset loss model to obtain a first loss result;

Inputting the processed sample image into a second preset loss model to obtain a second loss result;

And correcting parameters in the image processing model to be trained according to the first loss result and the second loss result to obtain a corrected image processing model.

4. A method according to claim 3, wherein the image processing model to be trained comprises a filter and an image class identification sub-model;

The method further comprises the steps of:

Performing Fourier transform processing on the sample image through the filter to obtain the direction characteristic of the sample image;

And carrying out image category identification processing on the direction features through the image category identification sub-model to obtain the category of the sample image.

5. The method of claim 4, wherein the image processing model to be trained further comprises an image reconstruction sub-model corresponding to a class of the sample image;

correspondingly, performing image processing on the sample image through the image processing model to be trained to obtain a processed sample image, including:

And carrying out image reconstruction processing on the sample image through the image reconstruction sub-model to obtain the processed sample image.

6. The method of claim 5, wherein the image reconstruction sub-model includes a feature extraction layer and an image reconstruction processing layer;

performing image reconstruction processing on the sample image through the image reconstruction sub-model to obtain the processed sample image, including:

Extracting the characteristics of the sample image through the characteristic extraction layer to obtain the image characteristics of the sample image;

and carrying out image reconstruction processing on the image features through the image reconstruction processing layer to obtain the processed sample image.

7. The method of claim 6, wherein the image reconstruction processing layer comprises a convolution layer and an upsampling layer;

The image reconstruction processing is carried out on the image features through the image reconstruction processing layer to obtain the processed sample image, and the method comprises the following steps:

Carrying out convolution processing on the image features through the convolution layer to obtain convolution processing features;

Performing upsampling processing on the convolution processing characteristics through the upsampling layer to obtain upsampling characteristics;

and determining the processed sample image according to the upsampling characteristics.

8. The method of claim 3, further comprising inputting a preset category corresponding to the sample image into the first preset loss model, and inputting a processed image corresponding to the sample image into the second preset loss model;

Correspondingly, the inputting the category of the sample image into a first preset loss model to obtain a first loss result includes:

Determining a first similarity between the category of the sample image and the preset category through the first preset loss model;

determining the first loss result according to the first similarity;

Inputting the processed sample image into a second preset loss model to obtain a second loss result, wherein the second loss result comprises:

Determining a second similarity between the processed sample image and the processed image through the second preset loss model;

And determining the second loss result according to the second similarity.

9. An image processing apparatus, comprising:

The device comprises a feature extraction module, a convolution neural network, a feature extraction module, a processing module and a processing module, wherein the feature extraction module is used for carrying out Fourier transform processing on an image to be processed by adopting a filter to obtain the direction feature of the image to be processed, wherein the direction feature is the local feature of a local area in the image to be processed;

The device comprises a local feature, an identification module, a processing module and a processing module, wherein the local feature is subjected to convolution processing and pooling processing in sequence to obtain pooled processing features;

The image reconstruction module is used for carrying out image reconstruction processing on the image to be processed by adopting an image processing model according to the type of the image to be processed and the image characteristics to obtain a processed image, wherein the image processing model comprises the filter, an image type recognition sub-model and an image reconstruction sub-model corresponding to the type;

And the output module is used for outputting the processed image.

10. An image processing apparatus, characterized by comprising:

a memory for storing executable instructions, and a processor for implementing the method of any one of claims 1 to 8 when the executable instructions stored in the memory are executed.

11. A computer readable storage medium storing executable instructions for causing a processor to perform the method of any one of claims 1 to 8.