CN114677267A - Image processing method, storage medium and terminal equipment - Google Patents

Abstract

The present application discloses an image processing method, a storage medium and a terminal device. The method includes acquiring an image to be processed, generating several sub-band images according to the to-be-processed image, and rearranging the several sub-band images to form a plurality of channel images. frequency band image; process the frequency band image to obtain an output image corresponding to the to-be-processed image. In the present application, the correlation of image data can be reduced by using the frequency band images corresponding to the images to be processed, thereby reducing spatial redundancy, thereby reducing the amount of calculation, reducing the computing power requirements of image processing on terminal equipment, and expanding the application of image processing methods. scope.

Description

An image processing method, storage medium and terminal device

technical field

The present application relates to the technical field of image processing, and in particular, to an image processing method, a storage medium and a terminal device.

Background technique

With the development of computer vision technology, people have more and more demands for image quality. In order to obtain high-quality images, image processing is often performed on images, such as image enhancement, image compression, and image super-score. However, the existing image processing methods generally require a lot of computing power and are difficult to deploy on mobile devices.

SUMMARY OF THE INVENTION

The technical problem to be solved by the present application is to provide an image processing method, a storage medium and a terminal device in view of the deficiencies of the prior art.

In order to solve the above technical problem, the first aspect of the embodiments of the present application provides an image processing method, the method includes:

acquiring an image to be processed, and generating several sub-band images according to the to-be-processed image;

Rearrange several sub-band images to form several channel band images;

The frequency band image is processed to obtain an output image corresponding to the to-be-processed image.

The image processing method, wherein the to-be-processed image is an uncompressed image, and the acquiring the to-be-processed image and generating several sub-band images according to the to-be-processed image is specifically:

Acquire an image to be processed, and perform frequency domain transformation on the to-be-processed image to obtain several sub-band images.

The image processing method, wherein the performing frequency domain transformation on the to-be-processed image to obtain several sub-band images specifically includes:

Divide the image to be processed into several image blocks, and perform frequency domain transformation on each image block in the several image blocks, so as to obtain image blocks after frequency domain transformation corresponding to each image block;

For each frequency-domain transformed image block, the frequency coefficients in the frequency-domain transformed image block are arranged in the channel direction to obtain several sub-band images.

In the image processing method, the image size of each image block in the several image blocks is the same, and the number of the several sub-band images is equal to the number of pixels in the image block.

The image processing method, wherein the image processing method is applied to an image processing model, and the processing of the frequency band image to obtain an output image corresponding to the to-be-processed image is specifically:

The image processing model determines an output image corresponding to the to-be-processed image based on the frequency band image.

The image processing method, wherein, before the image processing model determines the output image corresponding to the to-be-processed image based on the frequency band image, the method further includes:

Acquiring the task type corresponding to the image processing model, and determining the channel weights corresponding to each channel of the input item corresponding to the image processing model based on the task type;

The channel weights are configured in the image processing model, and the configured image processing model is used as an image processing model.

The image processing method, wherein, among the channel weights corresponding to each channel of the input item corresponding to the image processing model, some channels have a channel weight value of zero.

The image processing method, wherein the channel weights corresponding to each channel of the input items corresponding to the image processing model are determined based on the task type as follows:

Determine the channel weight set corresponding to the task type according to the preset correspondence between the task type and the channel weight set, wherein the channel weight set includes several channel weights, and the input items of the several channel weights and the image processing model include: One-to-one correspondence of each channel;

Based on the determined set of channel weights, the channel weights corresponding to each channel of the input item corresponding to the image processing model are determined.

In the image processing method, in the preset correspondence between task types and channel weight sets, the channel weight sets corresponding to different task types are different.

In the image processing method, the channel weight set is set according to the task type, or determined through a channel attention mechanism.

The image processing method, wherein the image processing model determines, based on the frequency domain image, the output image corresponding to the to-be-processed image specifically includes:

The image processing model determines a target channel image in the frequency image through its configured channel weight coefficient, wherein the weight coefficient corresponding to the target channel image is less than a preset threshold;

The image processing model determines the output image corresponding to the to-be-processed image based on the target channel image.

A second aspect of the embodiments of the present application provides an image processing device, the image processing device comprising:

an acquisition module, configured to acquire an image to be processed, generate several sub-band images according to the to-be-processed image, and rearrange the several sub-band images to form several channel band images;

A processing module, configured to process the frequency band image to obtain an output image corresponding to the to-be-processed image.

A third aspect of the embodiments of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to Steps in any of the image processing methods described above are implemented.

A fourth aspect of the embodiments of the present application provides a terminal device, which includes: a processor, a memory, and a communication bus; the memory stores a computer-readable program executable by the processor;

The communication bus implements connection communication between the processor and the memory;

When the processor executes the computer-readable program, the steps in any of the above image processing methods are implemented.

Beneficial effects: Compared with the prior art, the present application provides an image processing method, a storage medium and a terminal device. The method includes acquiring a to-be-processed image and determining a frequency band image corresponding to the to-be-processed image, wherein the The frequency band image includes several channels, and each channel in the several channels has different corresponding frequency bands; the frequency band image is processed to obtain an output image corresponding to the to-be-processed image. The present application can reduce the correlation of image data by using the frequency band images corresponding to the images to be processed, thereby reducing spatial redundancy, thereby reducing the amount of calculation, reducing the computing power requirements of image processing on terminal equipment, and expanding the application of image processing methods. scope.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present application more clearly, the following briefly introduces the drawings that are used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without conforming to the creative work.

FIG. 1 is a flowchart of an image processing method provided by the present application.

FIG. 2 is a schematic diagram of the principle of a process of determining a frequency band image in the image processing method provided by the present application.

FIG. 3 is a schematic diagram of the principle of the processing process of the image processing model in the image processing method provided by the present application.

FIG. 4 is a schematic structural diagram of an image processing apparatus provided by the present application.

FIG. 5 is a schematic structural diagram of a terminal device provided by the present application.

Detailed ways

The present application provides an image processing method, storage medium and terminal device. In order to make the purpose, technical solutions and effects of the present application clearer and clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application.

It will be understood by those skilled in the art that the singular forms "a", "an", "the" and "the" as used herein can include the plural forms as well, unless expressly stated otherwise. It should be further understood that the word "comprising" used in the specification of this application refers to the presence of stated features, integers, steps, operations, elements and/or components, but does not preclude the presence or addition of one or more other features, Integers, steps, operations, elements, components and/or groups thereof. It will be understood that when we refer to an element as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Furthermore, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. As used herein, the term "and/or" includes all or any element and all combination of one or more of the associated listed items.

It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. It should also be understood that terms, such as those defined in a general dictionary, should be understood to have meanings consistent with their meanings in the context of the prior art and, unless specifically defined as herein, should not be interpreted in idealistic or overly formal meaning to explain.

The inventor found through research that with the development of computer vision technology, people have more and more demands on image quality. In order to obtain high-quality images, image processing is often performed on images, such as image enhancement, image compression, and image super-score. However, the existing image processing methods generally require a lot of computing power, especially when applying neural network models for image processing operations (such as image classification, image segmentation, and image super-score, etc.), large resolutions (such as , 3000*4000, etc.) original RGB images as input items, resulting in a large amount of computation for the neural network model, which makes the neural network model consume a lot of computing power and is difficult to deploy on mobile devices.

In order to solve the above problem, in the embodiment of the present application, after the image to be processed is acquired, the image to be processed is preprocessed to generate a corresponding frequency band image, the frequency band image includes several channels, and each channel in the several channels corresponds to The frequency bands of the frequency bands are different from each other; the frequency band images are processed to obtain an output image corresponding to the to-be-processed image. The present application can reduce the correlation of image data by using the frequency band images corresponding to the images to be processed, thereby reducing spatial redundancy, thereby reducing the amount of calculation, reducing the computing power requirements of image processing on terminal equipment, and expanding the application of image processing methods. scope.

The content of the application will be further described below through the description of the embodiments in conjunction with the accompanying drawings.

FIG. 1 is a schematic flowchart of an image processing method provided in this embodiment. The method can be performed by an image processing apparatus, and the apparatus can be implemented by software and applied to a smart terminal such as a smart phone, a tablet computer, or a personal digital assistant. Referring to FIG. 1, the image processing method provided by this embodiment specifically includes:

S10. Acquire an image to be processed, generate several sub-band images according to the to-be-processed image, and rearrange the several sub-band images to form several channel band images.

Specifically, the to-be-processed image may be an uncompressed original image or a compressed compressed image. When the to-be-processed image is an uncompressed original image, the to-be-processed image is captured by an imaging system (eg, camera, video camera, under-screen camera, etc.), or may be other external devices (eg, digital cameras, etc.) and store the image in the terminal device, or it may be the image sent to the terminal device through the cloud. When the image to be processed is a compressed image, the image to be processed may be obtained by image coding, for example, the image to be processed may be an image compressed by using cosine transform DCT.

Each sub-band image in the several sub-band images corresponds to a frequency range, and the respective frequency ranges corresponding to each sub-channel image are different from each other. The frequency band image is formed by arranging several sub-band images. The image content is the same as the image content of the to-be-processed image, the frequency band image includes several channels, each sub-channel image corresponds to a channel of the channel image, and the channels corresponding to each sub-band image are different from each other.

In an implementation of this embodiment, as shown in FIG. 2 , the to-be-processed image is an uncompressed image, and the acquiring the to-be-processed image and generating several sub-band images according to the to-be-processed image is specifically:

The image to be processed is acquired, and the image to be processed is subjected to frequency domain transformation to obtain a frequency band image.

Specifically, the images to be processed are all images captured by an imaging system (eg, an under-screen camera, etc.), wherein the imaging system may be configured for the terminal device itself or for other devices. For example, the images to be processed are all landscape images obtained by photographing a landscape scene by a mobile phone equipped with an imaging system, and the landscape images all belong to the RGB color space. The frequency transform is used to convert the to-be-processed image from the spatial domain to the frequency domain, wherein the frequency domain transform may be DCT transform (Discrete Cosine Transform, discrete cosine transform), wavelet transform, Laplace transform and guided filters.

In an implementation of this embodiment, the frequency domain transform adopts DCT transform. After the image to be processed is divided into blocks, DCT transform is performed on each image block, so that the transformed value image blocks can be guaranteed to have correlation. Therefore, the correlation of image data in the image block can be reduced, and a large amount of image information can be represented with a small amount of data to reduce spatial redundancy. Therefore, the acquiring the image to be processed and performing frequency domain transformation on the image to be processed to obtain the frequency band image specifically includes:

Divide the image to be processed into several image blocks, and perform frequency domain transformation on each image block in the several image blocks, so as to obtain image blocks after frequency domain transformation corresponding to each image block;

Arrange each frequency coefficient in each frequency domain transformed image block according to the channel direction to obtain several sub-band images.

Specifically, the image block is a partial image area of the to-be-processed image, any two image blocks in several image blocks, the first image block in the two image blocks and the second image block in the two image blocks are not intersect, and the image areas corresponding to each of the several image blocks constitute the to-be-processed image. It can be understood that the image to be processed can be obtained after splicing several image blocks, in other words, the image size of the spliced image obtained by splicing several image blocks is the same as the image size of the image to be processed, and the image of the spliced image obtained by splicing. The content is the same as the image content of the image to be processed.

In an implementation manner of this embodiment, the image sizes corresponding to each image block in the several image blocks are the same, for example, the image size of each image block is 8*8 or the like. Correspondingly, the process of dividing the image to be processed into several image blocks may be: acquiring the image height and image width of the image to be processed; determining the common divisor of the image height and the image width, and based on the common divisor, The image to be processed is divided into several image widths whose side lengths are common divisors. Wherein, the common divisor may be one or more than one; when there are multiple common divisors, one common divisor may be randomly selected from the multiple common divisors as the side length of the image block, or, The greatest common divisor is used as the side length of the image block, or the least common divisor is selected as the side length of the image block, and so on. Of course, in practical applications, the several image blocks can also be obtained by dividing them by other methods, as long as several image blocks can be obtained by dividing them. A target image size is determined based on the common divisor of the length and the common divisor of the width, and the to-be-processed image is divided into several image blocks according to the target image size, so that the image size of each image block is equal to the target image size, etc. .

For example: the image size of the image to be processed is 1024*1024, the common divisor of 1024 is 8 as the side length of the image block, and the image to be processed is divided into several square image blocks with a side length of 8. The image size of each image block Both are 8*8, so that 16348 image blocks can be divided, and each image block obtained by dividing is a part of the image area of the image to be processed, and the image to be processed can be obtained by splicing 16348 image blocks.

The frequency coefficients in the image block after frequency domain transformation are obtained by performing frequency domain transformation on the pixel values of the pixel points in the image block. One-to-one correspondence. After obtaining the image blocks transformed in the frequency domain, the frequency coefficients in the transformed image blocks in the frequency domain are arranged according to the channels to obtain a DCT coefficient image, and the elements located in the same channel in the DCT coefficient image corresponding to each image block form a sub-channel image, so that the number of several sub-channel images is the same as the number of channels of the DCT coefficient image. The number of channels of the DCT coefficient image is the same as the number of pixels in the corresponding image block after frequency domain transformation, and the number of frequency coefficients in the image block after frequency domain transformation is equal to the number of pixels in the corresponding image block The number of points, thus, the number of channels of the DCT coefficient image is equal to the number of pixels in its corresponding image block, and each channel image in the DCT coefficient image contains only one frequency coefficient, in other words, the Arranging the frequency coefficients in the frequency-domain transformed image block in the channel direction refers to converting the M×N frequency-domain transformed image block into a 1×1×(M*N) DCT coefficient image. In an implementation of this embodiment, the image size of the image block is 8×8, that is, the image block includes 64 pixels, then the number of frequency coefficients in the frequency-domain transformed image block corresponding to the image block is 64, Correspondingly, the number of channels of the DCT coefficient image corresponding to the image block is 64, and the image scale of the DCT coefficient image is 1×1×64.

In an implementation of this embodiment, the frequency coefficient corresponding to each channel in the DCT coefficient image increases as the channel number increases, wherein the frequency coefficient is determined by the image block through frequency domain transformation, It is used to reflect the frequency corresponding to each pixel in the image block, and the size relationship may be a relationship from large to small, or a relationship from small to large. In a specific implementation of this embodiment, the size relationship is a relationship from small to large, and the arrangement of the frequency coefficients in the frequency domain transformed image block according to the size relationship in the channel direction may be: obtaining The number of pixels in the image block, and creating channels of the number of pixels, and arranging the frequency coefficients in the frequency domain transformed image block to each channel in ascending order to obtain a DCT coefficient image. Wherein, each channel of the DCT coefficient image includes a frequency coefficient, and for two adjacent channels, when the channel number of channel A in the two channels is greater than the channel number of channel B in the two channels, channel A The frequency coefficient in channel B is greater than or equal to the frequency coefficient in channel B. For example, if the image size of the image block is 2×2, and the frequency coefficients in the image block after frequency domain transformation are 0, 0, 2, and 2, respectively, the number of channels of the DCT coefficient image corresponding to the image block is 4, 4 The channel numbers of the channels are marked as 0, 1, 2 and 3 respectively. The frequency coefficient corresponding to channel number 0 is 0, the frequency coefficient corresponding to channel number 1 is 0, the frequency coefficient corresponding to channel number 2 is 2, and the frequency coefficient corresponding to channel number 3 is 2. .

In an implementation of this embodiment, the image size of each image block in several image blocks is the same, that is, the number of pixels included in each image block is the same, and correspondingly, the number of channels of the DCT coefficient image corresponding to each image block is the same. For example, if the number of pixels included in each image block is 64, then the number of channels of the DCT coefficient image corresponding to each image block is 64. Therefore, after several DCT coefficient images are acquired, a frequency band image is obtained by splicing several DCT coefficient images, the channel number of the frequency band image is equal to the channel number of each DCT coefficient image, and the position of each DCT coefficient image in the frequency band image The information corresponds to the position information of the image blocks corresponding to each frequency band image in the image to be processed. The position information of the image block in the image to be processed refers to the relative position of the image block in the image to be processed. When each image block is taken as a unit, the position information of the image block is the relative position of the image block in the image to be processed. The coordinate information of the image block in the image to be processed; the position information of the DCT coefficient image in the frequency band image is the relative position of the DCT coefficient image in the frequency band image, when each DCT coefficient image is regarded as a unit, the The position information of the DCT coefficient image is the coordinate information of the DCT coefficient image in the frequency band image. For example, for DCT coefficient image A and DCT coefficient image B, DCT coefficient image A corresponds to image block a, DCT coefficient image corresponds to image block b, and image block a is in the upper left corner of the image to be processed, then DCT coefficient image A is in the frequency band image. In the upper left corner, the image block b is in the upper right corner of the image to be processed, then the DCT coefficient image B is in the upper right corner of the frequency band image.

In an implementation manner of this embodiment, based on the acquisition of all DCT coefficient images, the process of determining the frequency band images corresponding to the to-be-processed images may be: after acquiring all the DCT coefficient images, acquiring the corresponding corresponding DCT coefficient images The position information of the image block in the image to be processed, and the position information of the image block corresponding to each DCT coefficient image is used as the position information corresponding to each DCT coefficient image, and each DCT coefficient image is based on the corresponding position information of each DCT coefficient image. Coefficient images are stitched to obtain band images. In addition, since the large frequency coefficients in the frequency domain transformed image block obtained by DCT transformation correspond to the image detail information in the image, and the small frequency coefficients correspond to the boundary information in the image, each channel in the frequency band image can be used for The image information of different positions of the image to be processed is reflected, wherein the channel corresponding to the large frequency coefficient can be used to reflect the image detail information, and the channel corresponding to the small frequency coefficient can be used to reflect the image boundary information. For example, the channel with a large channel number in the frequency band image is used to store large frequency coefficients, and the channel with a small channel number in the frequency band image is used to store small frequency coefficients, then the channel with a large channel number is used to reflect the image detail information, and the channel with a small channel number is used to store the image details. It is used to reflect the image boundary information. When processing the frequency band image, the channels in the frequency band image can be filtered based on the image content required by the processing type to remove some channels, which can further reduce the calculation amount of image processing.

For example, several image blocks include image block A, image block B, image block C and image block D, image block A corresponds to DCT coefficient image a, image block B corresponds to DCT coefficient image b, and image block C corresponds to DCT coefficient image c, The image block D corresponds to the DCT coefficient image d. When the image block is used as a unit, the position information of the image block A is (0, 0), the position information of the image block B is (0, 1), and the position information of the image block C is (0, 1). The information is (1,0), the position information of the image block D is (1,1), then the position information of the DCT coefficient image a in the frequency band image is (0,0), and the position information of the DCT coefficient image b is (0 , 1), the position information of the DCT coefficient image c is (1, 0), and the position information of the DCT coefficient image d is (1, 1).

S20. Process the frequency band image to obtain an output image corresponding to the to-be-processed image.

Specifically, the output image corresponds to the processing performed on the frequency band image. For example, when the processing is super-division processing, the output image is a super-division image; when the processing is face segmentation processing, the output image is A face image that can be carried for the image to be processed. In an implementation of this embodiment, the processing may include image classification, image segmentation, and image super-score.

In an implementation manner of this embodiment, the processing of the frequency band image may be performed by a trained image processing model. Correspondingly, the processing of the frequency band image to obtain the output image corresponding to the to-be-processed image is specifically:

The image processing model determines an output image corresponding to the to-be-processed image based on the frequency band image.

Specifically, the image processing model is a trained network model, such as an image super-segmentation model, an image segmentation model, and an image classification model. Wherein, in the training process of the image processing model, the training images in the training sample set corresponding to the image processing model are frequency band images. In other words, after the training sample set is acquired, for each to-be-processed image in the training sample set The image is subjected to frequency domain transformation to obtain a frequency band image corresponding to each image to be processed, and the frequency band image is input into the preset network model corresponding to the image processing model, and the preset network model is trained to obtain the image processing model. Model. The model structure of the preset network model is the same as the model structure of the image processing model, for example, both include a convolution layer, a fully connected layer, an activation layer, and the like. The model parameters of the preset network model are different from those of the image processing model, wherein the model parameters of the preset network model are initial model parameters, and the model parameters of the image processing model are model parameters obtained by training based on a training sample set. In addition, when training the preset network model, the adopted loss function may include spatial domain MSE (pixel L2 similarity) loss term, SSIM (image structure similarity) loss term, and frequency domain MSE (DCT coefficient L2 similarity) or, including spatial domain MSE (pixel L2 similarity) and frequency domain MSE (DCT coefficient L2 similarity) loss terms, or, including SSIM (image structure similarity) loss term and frequency domain MSE (DCT coefficient L2 similarity) loss term, etc.

In an implementation of this embodiment, the image processing model may be configured with a channel attention mechanism, and the image processing model may configure channel weights for each channel of each frequency image, and the channel attention mechanism may be based on the configuration of each channel. The channel weight is used to pay attention to the channel, so that the image channel required for the image processing task corresponding to the image processing model can be paid attention to, and the training speed of the image processing model can be improved during the training process of the image processing model. The data amount of the frequency band image can be reduced, thereby reducing the calculation amount.

Based on this, before the image processing model determines the output image corresponding to the to-be-processed image based on the frequency band image, the method further includes:

Acquiring the task type corresponding to the image processing model, and determining the channel weights corresponding to each channel of the input item corresponding to the image processing model based on the task type;

The channel weights are configured in the image processing model, and the configured image processing model is used as an image processing model.

Specifically, the task type is used to reflect the processing task to be performed by the image processing model, wherein the task type may include a super-resolution task, a denoising task, a segmentation task, a classification task, and the like. Among them, the image processing models corresponding to different task types have different channel weights for each channel. For example, when the task type is a classification task, since the classification task is more sensitive to the low-frequency channel data table, a high-channel weight can be configured for the low-frequency channel. Configure low-channel weights for high-frequency channel data; when the task type is super-division task, since super-division tasks are sensitive to high-frequency channels, high-channel weights can be configured for high-frequency channels, and low-channel weights can be configured for low-frequency channel data.

For example: the task type is a classification task, and the input item of the image processing model includes 4 channels, namely channel 0, channel 1, channel 2 and channel 3, where the frequencies corresponding to channel 0, channel 1, channel 2 and channel 3 are The coefficients increase sequentially, then the weight coefficient of channel 0 can be 1, the weight coefficient of channel 1 can be 0.8, the channel coefficient of channel 2 can be 0.6, and the channel coefficient of channel 3 can be 0.1.

In an implementation manner of this embodiment, the channel weight corresponding to each channel of the input item corresponding to the image processing model is determined based on the task type as follows:

Determine the channel weight set corresponding to the task type according to the preset correspondence between the task type and the channel weight set;

Based on the determined set of channel weights, the channel weights corresponding to each channel of the input item corresponding to the image processing model are determined.

Specifically, the corresponding relationship between the task type and the channel weight set is pre-configured. Based on the corresponding relationship between the task type and the channel weight set, a channel weight set corresponding to each task type can be obtained, and the channel weight set includes several channels. The weights of several channels are in one-to-one correspondence with each channel included in the input item of the image processing model. In addition, the channel weight sets corresponding to different task categories may be different, and a weight value of 0 may exist in the channel weight set corresponding to each task category. In other words, among the channel weights corresponding to each channel of the input item corresponding to the image processing model, the channel weight value of some channels is 0. In addition, the channel weight set corresponding to the task type may be set by preference, that is, the channel weight set is set according to the task type, or it may be determined by a channel attention mechanism, that is, by configuring a channel The network model of the attention mechanism learns the set of channel weights corresponding to the task.

In an implementation of this embodiment, the image processing model is preset with several configuration parameters, where the several configuration parameters may include a learning rate parameter, a reverse learning measurement parameter, a default channel weight parameter, and the like. After the model structure of the image processing model is determined, several configuration parameters corresponding to the image processing model need to be configured, and the image processing model configured with the configuration parameters is an image processing model. The channel weight parameter is used to configure the channel weight corresponding to the image processing model, so that the image processing model can determine the channel that needs attention based on the channel weight parameter. Therefore, configuring the channel weight in the image processing model is specifically: taking the channel weight corresponding to each channel of the input item corresponding to the acquired image processing model as a default channel weight parameter, and configuring the default channel weight parameter in the image processing model. The image processing model, so that the default channel weight parameter configured by the image processing model determines the channel weight corresponding to each channel of the input item, so that when the image processing model determines the output image corresponding to the frequency band image, the frequency band image is determined based on the default channel weight parameter. Each channel in the corresponding channel weight. It can be seen that the configuration process of the several channel weights is before the image processing model training process, and after configuring the several channel weights in the preset network model corresponding to the image processing model, based on the training image set, several channel weights and initial A preset network model of model parameters is trained to obtain an image processing model configured with the trained model parameters.

In an implementation manner of this embodiment, the image processing model determines, based on the frequency band image, the output image corresponding to the to-be-processed image specifically includes:

The image processing model determines a target channel image in the frequency image through its configured channel weight coefficient, wherein the weight coefficient corresponding to the target channel image satisfies a preset condition;

The image processing model determines the output image corresponding to the to-be-processed image based on the target channel image.

Specifically, the preset condition may be predetermined. When the weight coefficient corresponding to the target channel satisfies the preset condition, it indicates that the target channel is the channel concerned by the image processing model. When the weight coefficient corresponding to the target channel When the preset condition is not met, it is indicated that the target channel is not the channel concerned by the image processing model. In an implementation of this embodiment, the preset condition may be that the weight coefficient of the target channel is greater than the preset coefficient threshold, for example, the preset coefficient threshold is 0.5, and the frequency band image includes channel 0, channel 1 and channel 2, and the channel The weight coefficient corresponding to 0 is 0.8, the weight coefficient corresponding to channel 1 is 0.6, and the weight coefficient corresponding to channel 2 is 0.2, then channel 0 and channel 1 are target channels. In this embodiment, by selecting the target channel and ignoring the unselected channel data, the amount of data to be processed by the image processing image can be reduced, thereby reducing the computing power required by the image processing model, so that the image processing model can be arranged on the mobile terminal, At the same time, the processing speed of the image processing model can also be improved, and the real-time performance of the image processing can be improved.

In the first implementation manner of this embodiment, as shown in FIG. 3 , the image processing model may first perform pooling processing on the frequency band images to obtain the first feature map; After several full connection processing and activation processing, the probability of each channel of the frequency band image is obtained, and finally the output image is determined based on the probability of each channel and the frequency band image. When determining the probability of each channel, the probability corresponding to the channel that does not meet the preset conditions is 0. Based on this, the processing process of the image processing model to the frequency band image can be expressed as:

X=x*softmax(fc(pooling(x)))

Among them, pooling() represents pooling processing, fc() represents full connection processing, and softmax represents activation processing.

To sum up, this embodiment provides an image processing method, a storage medium, and a terminal device. The method includes acquiring an image to be processed, and dividing the to-be-processed image into several image blocks; frequency domain transformation, and determining the frequency band image corresponding to the to-be-processed image based on each frequency-domain transformed image block; and processing the frequency band image to obtain an output image corresponding to the to-be-processed image. The present application obtains the frequency band image corresponding to the image to be processed by performing frequency domain transformation on each image block, which can reduce the correlation of image data, thereby reducing spatial redundancy, thereby reducing the amount of calculation and reducing the computational power of image processing on the terminal device. requirements, to expand the scope of application of image processing methods.

Based on the above-mentioned image processing method, the present embodiment provides an image processing apparatus, as shown in FIG. 4 , the image processing apparatus includes:

an acquisition module 100, configured to acquire an image to be processed, generate several sub-band images according to the to-be-processed image, and rearrange the several sub-band images to form several channel band images;

The processing module 200 is configured to process the frequency band image to obtain an output image corresponding to the to-be-processed image.

In addition, it is worth noting that the working process of the image processing apparatus provided in this embodiment is the same as the working process of the above-mentioned image processing method, which will not be repeated here. For details, refer to the working process of the above-mentioned image processing method.

Based on the above image processing method, this embodiment provides a computer-readable storage medium, where the computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors , so as to realize the steps in the image processing method described in the above embodiments.

Based on the above image processing method, the present application also provides a terminal device, as shown in FIG. 5 , which includes at least one processor 20; a display screen 21; and a memory 22, which may also include a communication interface ( Communications Interface) 23 and bus 24. The processor 20 , the display screen 21 , the memory 22 and the communication interface 23 can communicate with each other through the bus 24 . The display screen 21 is set to display the user guide interface preset in the initial setting mode. The communication interface 23 can transmit information. The processor 20 may invoke logic instructions in the memory 22 to perform the methods in the above-described embodiments.

In addition, the above-mentioned logic instructions in the memory 22 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

As a computer-readable storage medium, the memory 22 may be configured to store software programs and computer-executable programs, such as program instructions or modules corresponding to the methods in the embodiments of the present disclosure. The processor 20 executes functional applications and data processing by running the software programs, instructions or modules stored in the memory 22, ie, implements the methods in the above embodiments.

The memory 22 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the terminal device, and the like. Additionally, memory 22 may include high-speed random access memory, and may also include non-volatile memory. For example, U disk, removable hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes, or temporary state storage medium.

In addition, the specific process of loading and executing the above-mentioned storage medium and the multiple instruction processor in the terminal device has been described in detail in the above-mentioned method, and will not be described one by one here.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still be The technical solutions described in the foregoing embodiments are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions in the embodiments of the present application.

Claims (14)

1. an image processing method, is characterized in that, described method comprises: acquiring images to be processed, generating several sub-band images according to the to-be-processed images, and rearranging the several sub-band images to form several channel band images; The frequency band image is processed to obtain an output image corresponding to the to-be-processed image. 2 . The image processing method according to claim 1 , wherein the image to be processed is an uncompressed image, and the obtaining of the image to be processed and generating several sub-band images according to the image to be processed are specifically: 2 . Acquire an image to be processed, and perform frequency domain transformation on the to-be-processed image to obtain several sub-band images. 3. The image processing method according to claim 2, wherein the performing frequency domain transformation on the to-be-processed image to obtain several sub-band images specifically comprises: Divide the image to be processed into several image blocks, and perform frequency domain transformation on each image block in the several image blocks, so as to obtain image blocks after frequency domain transformation corresponding to each image block; For each frequency-domain transformed image block, the frequency coefficients in the frequency-domain transformed image block are arranged in the channel direction to obtain several sub-band images. 4 . The image processing method according to claim 3 , wherein the image size of each image block in the several image blocks is the same, and the number of the several sub-band images is the same as the number of pixels in the image block. 5 . equal in quantity. 5. The image processing method according to any one of claims 1-4, wherein the image processing method is applied to an image processing model, and the frequency band image is processed to obtain the corresponding image of the to-be-processed image. The output image is specifically: The image processing model determines an output image corresponding to the to-be-processed image based on the frequency band image. 6 . The image processing method according to claim 5 , wherein, before the image processing model determines the output image corresponding to the to-be-processed image based on the frequency band image, the method further comprises: 7 . Acquiring the task type corresponding to the image processing model, and determining the channel weights corresponding to each channel of the input item corresponding to the image processing model based on the task type; The channel weights are configured in the image processing model, and the configured image processing model is used as an image processing model. 7 . The image processing method according to claim 6 , wherein, among the channel weights corresponding to each channel of the input item corresponding to the image processing model, the channel weight value of some channels is zero. 8 . 8. The image processing method according to claim 7, wherein the channel weights corresponding to each channel of the input items corresponding to the image processing model determined based on the task type are specifically: Determine the channel weight set corresponding to the task type according to the preset correspondence between the task type and the channel weight set, wherein the channel weight set includes several channel weights, and the input items of the several channel weights and the image processing model include: One-to-one correspondence of each channel; Based on the determined channel weight set, the channel weight corresponding to each channel of the input item corresponding to the image processing model is determined. 9 . The image processing method according to claim 8 , wherein in the preset correspondence between task types and channel weight sets, the channel weight sets corresponding to different task types are different. 10 . 10 . The image processing method according to claim 8 , wherein the channel weight set is set according to the task type, or determined by a channel attention mechanism. 11 . 11. The image processing method according to claim 8, wherein the image processing model, based on the frequency domain image, determines the output image corresponding to the to-be-processed image specifically comprising: The image processing model determines a target channel image in the frequency image through its configured channel weight coefficient, wherein the weight coefficient corresponding to the target channel image is less than a preset threshold; The image processing model determines the output image corresponding to the to-be-processed image based on the target channel image. 12. An image processing device, wherein the image processing device comprises: an acquisition module, configured to acquire images to be processed, generate several sub-band images according to the to-be-processed images, and rearrange the several sub-band images to form several channel band images; A processing module, configured to process the frequency band image to obtain an output image corresponding to the to-be-processed image. 13. A computer-readable storage medium, characterized in that the computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to realize the claim The steps in the image processing method described in any one of requirements 1-11. 14. A terminal device, comprising: a processor, a memory, and a communication bus; the memory stores a computer-readable program executable by the processor; The communication bus implements connection communication between the processor and the memory; When the processor executes the computer-readable program, the steps in the image processing method according to any one of claims 1-11 are implemented.

Images