CN118469850A

CN118469850A - Image processing method and device

Info

Publication number: CN118469850A
Application number: CN202410547152.7A
Authority: CN
Inventors: 汤成熙
Original assignee: Vivo Mobile Communication Co Ltd
Current assignee: Vivo Mobile Communication Co Ltd
Priority date: 2024-05-06
Filing date: 2024-05-06
Publication date: 2024-08-09

Abstract

The application discloses an image processing method and device, and belongs to the technical field of image processing. The processing method comprises the following steps: performing noise detection on a first image to be processed to obtain first noise information; determining second noise information based on the first noise information and a noise adjustment weight value, wherein the noise adjustment weight value is used to indicate a noise adjustment strength of at least one image region of the first image; inputting the first image and the second noise information into a first image processing network model, and acquiring a first noise reduction image and a first edge image generated by the first image processing network model according to the first image and the second noise information; and generating a processed second image according to the first noise reduction image and the first edge image.

Description

Image processing method and device

Technical Field

The application belongs to the technical field of image processing, and particularly relates to an image processing method and device.

Background

In the related art, image noise reduction is an important bottom image processing technology, and aims to remove random useless signals generated due to signal interference and environmental effects in the image acquisition and transmission processes, improve image quality, and influence task accuracy of subsequent high-level semantic recognition and the like due to noise reduction effects. The traditional method comprises a three-dimensional Block matching algorithm (Block-MATCHING AND 3D filtering,BM3D), and the BM3D groups by searching similar blocks of the image and performs collaborative filtering noise reduction, so that the computational complexity is high, and the noise residue is more.

Although the noise reduction method based on the depth convolution neural network can well remove noise on the image, excessive noise reduction is easy to cause loss of detail textures of the image.

Disclosure of Invention

The embodiment of the application aims to provide an image processing method and device, which can solve the problem that the detail texture of an image is lost due to excessive noise reduction in the image noise reduction method in the prior art.

In a first aspect, an embodiment of the present application provides a method for processing an image, including:

performing noise detection on a first image to be processed to obtain first noise information;

Determining second noise information based on the first noise information and a noise adjustment weight value, wherein the noise adjustment weight value is used to indicate a noise adjustment strength of at least one image region of the first image;

inputting the first image and the second noise information into a first image processing network model, and acquiring a first noise reduction image and a first edge image generated by the first image processing network model according to the first image and the second noise information;

A second image is generated from the first noise reduction image and the first edge image.

In a second aspect, an embodiment of the present application provides an image processing apparatus, including:

The detection module is used for carrying out noise detection on the first image to be processed to obtain first noise information;

A determining module for determining second noise information based on the first noise information and a noise adjustment weight value, wherein the noise adjustment weight value is used for indicating a noise adjustment strength of at least one image area of the first image;

The processing module is used for inputting the first image and the second noise information into the first image processing network model, and acquiring a first noise reduction image and a first edge image which are generated by the first image processing network model according to the first image and the second noise information;

and the generating module is used for generating a second image according to the first noise reduction image and the first edge image.

In a third aspect, embodiments of the present application provide an electronic device comprising a processor and a memory storing a program or instructions executable on the processor, the program or instructions implementing the steps of the method as in the first aspect when executed by the processor.

In a fourth aspect, embodiments of the present application provide a readable storage medium having stored thereon a program or instructions which when executed by a processor perform the steps of the method as in the first aspect.

In a fifth aspect, embodiments of the present application provide a chip comprising a processor and a communication interface coupled to the processor for running a program or instructions implementing the steps of the method as in the first aspect.

In a sixth aspect, embodiments of the present application provide a computer program product stored in a storage medium, the program product being executable by at least one processor to implement a method as in the first aspect.

In the embodiment of the application, when the first image to be processed is subjected to noise reduction processing, the noise adjustment weight is combined to determine the second noise information of the first image to be processed, the second noise information can reflect the intensity of noise reduction required by a user on the first image, the first image to be processed and the second noise information are input into the first image processing network model together, so that the first image processing network model can perform noise reduction processing according to the noise reduction intensity required by the user, excessive noise reduction is avoided, meanwhile, the first image processing network model also has edge detection capability, the first edge image is output while the first noise reduction image after noise reduction is output, the first edge image can retain the texture information of the original image, and the second image with good noise reduction effect and more texture information is retained can be obtained by combining the first noise reduction image and the first edge image after noise reduction, so that the problem of losing the detail texture of the image due to excessive noise reduction is solved.

Drawings

FIG. 1 illustrates a flow chart of a method of processing an image in accordance with some embodiments of the application;

FIG. 2 illustrates a flow chart of a method of processing an image in accordance with some embodiments of the application;

FIG. 3 illustrates a flow chart of a method of processing an image in accordance with some embodiments of the application;

FIG. 4 illustrates a schematic diagram of a first image processing network model according to some embodiments of the application;

FIG. 5 illustrates a flow chart of a method of processing an image in accordance with some embodiments of the application;

FIG. 6 illustrates a block diagram of an image processing apparatus of some embodiments of the application;

FIG. 7 shows a block diagram of an electronic device according to an embodiment of the application;

Fig. 8 is a schematic diagram of a hardware structure of an electronic device implementing an embodiment of the present application.

Detailed Description

The technical solutions of the embodiments of the present application will be clearly described below with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which are obtained by a person skilled in the art based on the embodiments of the present application, fall within the scope of protection of the present application.

The terms first, second and the like in the description and in the claims, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments of the application are capable of operation in sequences other than those illustrated or otherwise described herein, and that the objects identified by "first," "second," etc. are generally of a type not limited to the number of objects, for example, the first object may be one or more. Furthermore, in the description and claims, "and/or" means at least one of the connected objects, and the character "/", generally means that the associated object is an "or" relationship.

The method and the device for processing the image provided by the embodiment of the application are described in detail below through specific embodiments and application scenes thereof with reference to the accompanying drawings.

In some embodiments of the present application, there is provided an image processing method, fig. 1 shows a flowchart of the image processing method of some embodiments of the present application, and as shown in fig. 1, the processing method includes:

Step 102, noise detection is performed on the first image to be processed, and first noise information is obtained.

In the embodiment of the present application, the first image is specifically an image that needs to be subjected to noise reduction processing. In some cases, noise may be included in the image, and the noise may be an unwanted signal generated during the image acquisition process due to environmental effects, or may be an unwanted signal generated during the image transmission process due to signal interference. These noise signals may appear in the image in the form of "noise points" that affect the display of the image.

The noise detection is performed on the first image including the noise to obtain first noise information, which is specifically information capable of reflecting the noise signal level of the first image. Illustratively, the noise level of the first image may be estimated by statistically detecting noise of the first image. Illustratively, the flat area of the first image is found by counting the gradients of the first image, and the pixel values of the flat area of the first image are counted to obtain a standard deviation, which is the first noise information reflecting the noise level of the first image.

Step 104, determining second noise information based on the first noise information and a noise adjustment weight value, wherein the noise adjustment weight value is used for indicating noise adjustment intensity of at least one image area of the first image.

In the embodiment of the application, the first noise information is obtained through noise detection, and can objectively reflect the data of the actual noise level of the first image to a certain extent.

In the embodiment of the application, after the first noise information of the first image is detected, the second noise information is determined by combining the noise adjustment weight value, and the noise adjustment weight value can be set by a user or can be obtained by identifying the image content of the first image. The noise adjustment weight value is specifically used to indicate noise adjustment weights in one or more image areas in the first image, and it can be understood that the higher the noise adjustment weight is, the greater the intensity representing the noise reduction process.

And adjusting the identified first noise information by setting a noise adjustment weight value, wherein the obtained second noise information is information for indicating the noise reduction intensity of the first image processing network model on each image area in the first image. By the method, the problem of texture detail loss caused by excessive noise reduction can be effectively avoided.

And step 106, inputting the first image and the second noise information into a first image processing network model, and acquiring a first noise reduction image and a first edge image generated by the first image processing network model according to the first image and the second noise information.

In an embodiment of the present application, the first image processing network model is a pre-trained first neural network model, and the network structure of the first image processing network model may be an encoder-decoder structure, for example. The network input of the first image processing network model includes a first image to be processed and second noise information indicating a noise reduction processing intensity for each image region in the first image. By adding the second noise information in the network input, the dynamic adjustment of the noise reduction intensity can be realized, and excessive noise reduction is avoided.

The output data of the first image processing network model comprises a first noise reduction image and a first edge image, wherein the first noise reduction image is a noise reduction clean image predicted by the first image processing network model, and the first edge image is an edge image of the clean image predicted by the first image processing network model, and the edge image comprises edge images and texture information of an image object contained in the first image.

When the image is enhanced, the traditional sharpening algorithm can make the image texture clearer, but can enhance the image noise at the same time, so that the purity of the picture is reduced. The first image processing network model of the embodiment of the application can simultaneously output the first noise reduction image after noise reduction and the sharpened first edge image, so that the image texture can be clear, and the image noise can be reduced.

Step 108, generating a second image according to the first noise reduction image and the first edge image.

In the embodiment of the application, the first noise reduction image is a clean image after noise reduction treatment, the first edge image is an edge image obtained after edge enhancement of the clean image, and the first noise reduction image and the first edge image are combined to obtain a final second image after treatment, so that good noise signal removal effect and clear image texture can be ensured.

When the first image to be processed is subjected to noise reduction processing, the second noise information of the first image to be processed is determined by combining the noise adjustment weight, the second noise information can reflect the intensity of noise reduction required by a user on the first image, the first image to be processed and the second noise information are input into the first image processing network model together, so that the first image processing network model can perform noise reduction processing according to the noise reduction intensity required by the user, excessive noise reduction is avoided, meanwhile, the first image processing network model also has edge detection capability, a first edge image is output while a first noise reduction image after noise reduction is output, the first edge image can retain texture information of an original image, and a second image with good noise reduction effect and more texture information is retained by combining the first noise reduction image and the first edge image, so that the problem of image detail texture loss caused by excessive noise reduction is solved.

In some embodiments of the present application, fig. 2 shows a flowchart of a method of processing an image according to some embodiments of the present application, as shown in fig. 2, before inputting the first image and the second noise information into the first image processing network model, the method further includes:

Step 202, a third image is acquired.

In the embodiment of the present application, the third image is a clean image, that is, an image containing no noise information. The third image may be an image photographed by a cellular phone, a digital camera, or a video camera, for example. The third image may be, for example, an image collected from a public image database.

And step 204, adding noise data to the third image based on the third noise information to obtain a noise image.

In the embodiment of the application, a training image data set of a large number of clean images-noise images is manufactured in a synthetic mode by modeling the noise model. By adding noise data to the third image, a noise image can be obtained. Illustratively, the method of generating the noise image is as follows as shown in equation (1):

X＝Y+n；(1)

wherein X is a noise image, Y is a third image, and n is noise data.

Illustratively, the noise data is obtained by gaussian distribution modeling based on the third noise information. Illustratively, the noise data n satisfies n to Gaussian (0, σ), where σ is a noise standard deviation, that is, the above-mentioned third noise information, and by setting different third noise information, a training data pair of a third image-noise image with different noise levels can be obtained.

And 206, performing edge detection on the third image to obtain a second edge image.

In the embodiment of the present application, in addition to the training data pair of the third image-noise image, edge detection is performed on the third image in the training data, so as to extract an edge image of the clean image, that is, the second edge image. Illustratively, the third image is subjected to edge detection by using a laplace edge detection method to obtain a second edge image.

Step 208, generating training data according to the third image, the third noise information, the noise image and the second edge image.

In the embodiment of the application, training data pairs of a third image and a noise image, third noise information when the noise data is added to the third image, and a second edge image obtained by performing edge detection on a clean third image are arranged into training data, and are used for training a preset first neural network model.

Step 210, training the first neural network model through the training data and the target loss function to obtain a first image processing network model.

In the embodiment of the application, in the training process, the network input of the first neural network model comprises two parts, namely a noise image and third noise information. The third noise information is made into a map of the same size as the noise image to identify the noise level size added by the current noise image. The data is randomly extracted from the data at different noise levels during the whole training process as input to the network. The input form can be multiple inputs, the characteristics are combined after being respectively extracted by different characteristic extraction modules of the network, and the characteristics can be directly combined into the same tensor by a combination means and then input into the network.

And obtaining a corresponding output image and an output edge through calculation of the first neural network model, taking a clean third image and a second edge image as supervision, and training the first neural network model by applying a target loss function until the target loss function converges, so as to obtain a trained first image processing network model.

Fig. 3 shows a flow chart of a method of processing an image according to some embodiments of the application, as shown in fig. 3, the method comprising:

step 302, noise and edge training data pairs are generated.

The data generated in this step is used to train the first neural network model.

Step 304, training a dual input/output network model.

Wherein the dual input includes a noise image and noise information and the dual output includes a clean image and an edge image.

In step 306, the sharpness is dynamically adjusted for edge enhancement.

After the first image processing network model is obtained through training, edge enhancement of the noise image and dynamic adjustment of definition can be achieved through the first image processing network model.

In some embodiments, the neural network model employs a dual encoder network-dual decoder network architecture, with two sets of encoder-decoders for processing two different inputs and outputs, respectively, namely image noise reduction and edge detection.

Illustratively, the first encoder network and the first decoder network form a noise processing network, the network input of the first encoder network is a noise image, and after the network processing, the first decoder network outputs a clean image after the noise reduction processing. The second encoder network and the second decoder network form an edge detection network, the network input of the second encoder network is a clean image output by the first decoder network, and the network input of the second encoder network is a third edge image.

The structure of the double encoder network and the double decoder network has better processing effect, and can obtain processed images with better noise reduction effect and clearer edges.

In other embodiments, the neural network model adopts a structure of a single encoder network-a single decoder network, and combines and separates input and output by splicing and cutting. The design can be directly adapted and modified on the existing noise reduction network model structure, is simple to change, is improved on the existing noise reduction network, and has the advantage of strong applicability.

In addition, different network models can be flexibly selected according to actual conditions, a simple convolutional neural network model can be selected when the operation efficiency is emphasized, and a novel network model based on a transducer module can be selected when the effect is emphasized.

Fig. 4 shows a schematic diagram of a first image processing network model according to some embodiments of the present application, as shown in fig. 4, where the first image processing network model includes an input layer 402, an inference layer 404, and an output layer 406, the input layer being configured to input a noise image X and corresponding second noise information σ ₁, the inference layer being configured to infer by the network model, and the output layer being configured to output a first noise reduction image Y ₁ and a first edge image E ₁.

According to the embodiment of the application, the training data set is designed to comprise the training data pair of the clean image-noise image, the noise information of the noise image and the edge detection result of the clean image, and the first neural network model is trained through the training data set, so that the first image processing network model obtained through training has high-efficiency noise adjustment capability and edge detection capability.

In some embodiments of the present application, the objective loss function is as shown in the following equation (2):

Wherein L ₀ is a target loss function, n is the number of training data sets, Y (x, Y) is a pixel value of (x, Y) in a pixel coordinate of a noise image, Y ₁ (x, Y) is a pixel value of (x, Y) in a second noise reduction image output by the first neural network model based on the noise image, E (x, Y) is a pixel value of (x, Y) in a second edge image, and E ₁ (x, Y) is a pixel value of (x, Y) in a third edge image output by the first neural network model based on the noise image.

In the embodiment of the application, the target loss function is realized through a third image and a model output image and a mean-square error (MSE) of a second edge image and a model output edge image, and the training efficiency of the first image processing network model can be improved through the target loss function shown in the formula (2).

In some embodiments of the present application, before determining the second noise information based on the first noise information and the noise adjustment weight value, the processing method further includes:

Receiving noise reduction weight input by a user; and determining a noise adjustment weight value according to the noise reduction weight.

In the embodiment of the present application, the noise reduction weight input may be an input that a user performs noise reduction weight setting on the whole first image, or may be an input that a user performs noise reduction weight setting on at least one image area in the first image.

Specifically, for different images or different areas in the same image, the requirements of the user on the noise reduction intensity are different, for example, for a solid color area with large area such as sky and water surface and without complex textures, the higher noise reduction force can effectively improve the purity of the pictures, and for an area with complex textures and complex colors such as characters and dense plants, the original texture details and edge details may be lost if the noise reduction force is too large. Therefore, the user can freely set different noise reduction weights according to different conditions, so that the noise level of the local part or the whole picture is dynamically adjusted.

Illustratively, assuming that the noise adjustment weight value set by the user is w ₁ and the first noise information is σ, the second noise information can be obtained by the following formula (3):

σ₁＝σ×w₁；(3)

Wherein σ ₁ is second noise information, σ is first noise information, w ₁ is a noise adjustment weight value, the value range of w ₁ is [0,1], w ₁ is 0, no noise reduction is performed, and w ₁ is 1, which indicates that the noise reduction intensity is maximum.

According to the embodiment of the application, the noise reduction degree is determined by adjusting the weight value according to the noise input by the user, so that the noise reduction effect and the preservation of texture details can be considered for the image after the noise reduction treatment, and the actual requirements of the user can be met.

Carrying out semantic segmentation processing on the first image to obtain the region type of at least one image region; noise adjustment weight values are determined based on region type.

In the embodiment of the application, for different image contents, if the same noise reduction force is adopted for noise reduction processing, the obtained results may be different. For example, for large-area image contents such as sky and water surface without complex textures, the purity of the images can be effectively improved by adopting stronger noise reduction force, while for image contents such as characters, dense plants and the like with complex textures and complex colors, if the same stronger noise reduction force is adopted, fine textures, edge loss and loss of image details can be caused.

In view of the above problems, in the embodiment of the present application, by performing semantic segmentation on a first image to be processed, the region types corresponding to different image regions are determined, where the region types may be, for example, "solid color region", "complex texture region", or "portrait region", "background region", etc.

Different noise adjustment intensities, namely different noise adjustment weight values, are adopted for different region types, so that clean pictures are obtained while as much picture details as possible are reserved.

For example, assuming that the determined noise adjustment weight value corresponding to the region type is w ₁ and the first noise information is σ, the second noise information may be obtained by the following formula (4):

σ₁＝σ×w₁；(4)

According to the embodiment of the application, the region types of each region in the image to be processed are determined through semantic recognition, and the noise adjustment weight value is set according to different region types, so that the image after noise reduction processing can be given consideration to the noise reduction effect and the reservation of texture details, and the picture quality is improved.

In some embodiments of the application, the first image processing network model comprises an encoder network; before inputting the first image and the second noise information into the first image processing network model, the method further comprises:

performing splicing processing on the first image and the second noise information to obtain an input vector; the input vector is input to an encoder network.

In the embodiment of the present application, the neural network model includes an encoder network, and the encoder network and the decoder network are combined to form a codec network structure, and a common noise reduction network model is generally the same as the encoder network-decoder network, so that the first image processing network model can be obtained by modifying the existing noise reduction network model.

And according to the data input format of the encoder network, performing splicing processing on the first image and the second noise information to obtain a first vector conforming to the data input format of the encoder network, and after the first vector is input into the encoder network, a first image processing network model of the encoder network-decoder network structure can infer to obtain a clean first noise reduction image and a first edge image of the clean image.

The embodiment of the application combines and separates the input and the output in a splicing and cutting mode, so that the first image processing network model can be directly adapted and modified on the existing noise reduction network model structure, and the applicability is strong.

In some embodiments of the present application, generating a processed second image from a first noise reduction image and a first edge image includes:

Determining a second image by the following formula (5);

Y₂＝((Y₁-X)×w₂+X)+(E₁×w₃)；(5)

wherein Y ₂ is a second image, Y ₁ is a first noise reduction image, X is a first image, w ₂ is a noise reduction parameter, E ₁ is a first edge image, w ₃ is an edge enhancement parameter, the noise reduction parameter and the edge enhancement parameter are built-in parameters of a model, the noise reduction parameter and the edge enhancement parameter can be dynamically conditioned according to user requirements, and the range of values of the noise reduction parameter and the edge enhancement parameter is [0,1]. when w ₂ is 0, it means that noise is not reduced, and when w ₂ is 1, it means that noise reduction effect is the strongest. when w ₃ is 0, it means that edge enhancement is not performed, and when w ₃ is 1, it means that the edge enhancement effect is the strongest.

In the embodiment of the application, after a first image needing noise reduction processing and second noise information for indicating noise reduction intensity when the first image is subjected to noise reduction are input into a first image processing network model, a first noise reduction image and a first edge image output by the first image processing network model are obtained, wherein the first noise reduction image is a clean image after noise reduction, and the first edge image is an image generated after the clean image is subjected to edge detection processing. After the first noise reduction image and the first edge image are obtained, the first noise reduction image and the first edge image are fused through the formula (5), and a final second image is obtained.

Fig. 5 shows a flowchart of a method of processing an image according to some embodiments of the application, as shown in fig. 5, the method comprising:

step 502, the noise image is input to a noise estimation module.

The noise estimation module is used for determining first noise information corresponding to the noise image.

Step 504, the noise image and the noise information are input to the noise adjustment module.

The noise adjustment module is used for adjusting the first noise information based on the noise adjustment weight value obtained by user input or semantic recognition to obtain second noise information used for indicating noise reduction intensity.

Step 506, inputting the noise image and the second noise information to the network reasoning module.

The network reasoning module conducts reasoning noise reduction on the noise image based on the second noise information to obtain a clean image and an edge image of the clean image.

Step 508, the clean image and the edge image are input to the sharpness adjustment module.

The definition adjusting module fuses the clean image and the definition adjusting model based on parameters set in the network to obtain a clean and clear-texture output image.

According to the embodiment of the application, the clean image and the edge image of the clean image are predicted through the first image processing network model, so that the defect that the image noise is amplified while the image edge is enhanced by the traditional image sharpening method is avoided, the defect that the picture details are lost by the traditional noise reduction algorithm is avoided, the image details can be well recovered while the image noise is reduced, and the problem of losing the image details is reduced.

According to the image processing method provided by the embodiment of the application, the execution subject can be an image processing device. In the embodiment of the present application, an image processing apparatus provided in the embodiment of the present application will be described by taking an example of a method for executing an image processing by an image processing apparatus.

In some embodiments of the present application, there is provided an image processing apparatus, fig. 6 shows a block diagram of the image processing apparatus of some embodiments of the present application, and as shown in fig. 6, a processing apparatus 600 includes:

The detection module 602 is configured to perform noise detection on a first image to be processed to obtain first noise information;

A determining module 604 for determining second noise information based on the first noise information and a noise adjustment weight value, wherein the noise adjustment weight value is used to indicate a noise adjustment strength of at least one image region of the first image;

The processing module 606 is configured to input the first image and the second noise information into the first image processing network model, and obtain a first noise reduction image and a first edge image generated by the first image processing network model according to the first image and the second noise information;

a generating module 608 is configured to generate a second image according to the first noise reduction image and the first edge image.

In some embodiments of the application, the processing device further comprises:

the acquisition module is used for acquiring a third image;

the processing module is further used for adding noise data to the third image based on the third noise information to obtain a noise image;

the detection module is used for carrying out edge detection on the third image to obtain a second edge image;

the generating module is also used for generating training data according to the third image, the third noise information, the noise image and the second edge image;

and the training module is used for training the first neural network model through the training data and the target loss function to obtain a first image processing network model.

In the embodiment of the application, the target loss function is realized through the mean square error of the third image and the model output image and the second edge image and the model output edge image, and the training efficiency of the first image processing network model can be improved through the target loss function.

In some embodiments of the present application, a receiving module is configured to receive a noise reduction weight input of a user;

The determining module is further configured to determine a noise adjustment weight value in response to the noise reduction weight input.

In some embodiments of the present application, the processing module is further configured to perform semantic segmentation processing on the first image to obtain an area type of at least one image area;

The determining module is further used for determining a noise adjustment weight value based on the region type.

In some embodiments of the application, the first image processing network model comprises an encoder network;

The processing module is also used for performing splicing processing on the first image and the second noise information to obtain an input vector; the input vector is input to an encoder network.

In some embodiments of the application, the determining module is further configured to:

Determining a second image by the following formula (5);

Y₂＝((Y₁-X)×w₂+X)+(E₁×w₃)；(5)

Wherein Y ₂ is the second image, Y ₁ is the first noise reduction image, X is the first image, w ₂ is the noise reduction parameter, E ₁ is the first edge image, and w ₃ is the edge enhancement parameter.

The image processing device in the embodiment of the application can be an electronic device or a component in the electronic device, such as an integrated circuit or a chip. The electronic device may be a terminal, or may be other devices than a terminal. The electronic device may be a Mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted electronic device, a Mobile internet appliance (Mobile INTERNET DEVICE, MID), an augmented reality (augmented reality, AR)/Virtual Reality (VR) device, a robot, a wearable device, an ultra-Mobile personal computer (UMPC), a netbook or a Personal Digital Assistant (PDA), etc., and may also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a Television (TV), a teller machine, a self-service machine, etc., which are not particularly limited in the embodiments of the present application.

The image processing device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, and the embodiment of the present application is not limited specifically.

The image processing device provided by the embodiment of the present application can implement each process implemented by the above method embodiment, and in order to avoid repetition, details are not repeated here.

Optionally, an embodiment of the present application further provides an electronic device, fig. 7 shows a block diagram of a structure of the electronic device according to an embodiment of the present application, as shown in fig. 7, an electronic device 700 includes a processor 702, a memory 704, and a program or an instruction stored in the memory 704 and capable of running on the processor 702, where the program or the instruction implements each process of the foregoing method embodiment when executed by the processor 702, and the process can achieve the same technical effect, and is not repeated herein.

The electronic device in the embodiment of the application includes the mobile electronic device and the non-mobile electronic device.

The electronic device 800 includes, but is not limited to: radio frequency unit 801, network module 802, audio output unit 803, input unit 804, sensor 805, display unit 806, user input unit 807, interface unit 808, memory 809, and processor 810.

Those skilled in the art will appreciate that the electronic device 800 may also include a power source (e.g., a battery) for powering the various components, which may be logically connected to the processor 810 by a power management system to perform functions such as managing charge, discharge, and power consumption by the power management system. The electronic device structure shown in fig. 8 does not constitute a limitation of the electronic device, and the electronic device may include more or less components than shown, or may combine certain components, or may be arranged in different components, which are not described in detail herein.

The processor 810 is configured to perform noise detection on a first image to be processed to obtain first noise information; determining second noise information based on the first noise information and a noise adjustment weight value, wherein the noise adjustment weight value is used to indicate a noise adjustment strength of at least one image region of the first image; inputting the first image and the second noise information into a first image processing network model, and acquiring a first noise reduction image and a first edge image generated by the first image processing network model according to the first image and the second noise information; and generating a processed second image according to the first noise reduction image and the first edge image.

Optionally, the processor 810 is further configured to acquire a third image; adding noise data to the third image based on the third noise information to obtain a noise image; performing edge detection on the third image to obtain a second edge image; generating training data according to the third image, the third noise information, the noise image and the second edge image; and training the first neural network model through the training data and the target loss function to obtain a first image processing network model.

Optionally, the target loss function is as shown in the following equation (2):

Optionally, a user input unit 807 for receiving noise reduction weight input from a user;

The processor 810 is further configured to determine a noise adjustment weight value in response to the noise reduction weight input.

Optionally, the processor 810 is further configured to perform semantic segmentation processing on the first image to obtain a region type of at least one image region; noise adjustment weight values are determined based on region type.

Optionally, the first image processing network model comprises an encoder network;

the processor 810 is further configured to perform a stitching process on the first image and the second noise information to obtain an input vector; the input vector is input to an encoder network.

Optionally, the processor 810 is further configured to determine the second image by the following formula (5);

Y₂＝((Y₁-X)×w₂+X)+(E₁×w₃)；(5)

It should be appreciated that in embodiments of the present application, the input unit 804 may include a graphics processor (Graphics Processing Unit, GPU) 8041 and a microphone 8042, with the graphics processor 8041 processing image data of still pictures or video obtained by an image capturing device (e.g., a camera) in a video capturing mode or an image capturing mode. The display unit 806 may include a display panel 8061, and the display panel 8061 may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like. The user input unit 807 includes at least one of a touch panel 8071 and other input devices 8072. Touch panel 8071, also referred to as a touch screen. The touch panel 8071 may include two parts, a touch detection device and a touch controller. Other input devices 8072 may include, but are not limited to, a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and so forth, which are not described in detail herein.

The memory 809 can be used to store software programs as well as various data. The memory 809 may mainly include a first storage area storing programs or instructions and a second storage area storing data, wherein the first storage area may store an operating system, application programs or instructions (such as a sound playing function, an image playing function, etc.) required for at least one function, and the like. Further, the memory 809 may include volatile memory or nonvolatile memory, or the memory 809 may include both volatile and nonvolatile memory. The nonvolatile memory may be a Read-only memory (ROM), a programmable Read-only memory (ProgrammableROM, PROM), an erasable programmable Read-only memory (ErasablePROM, EPROM), an electrically erasable programmable Read-only memory (ElectricallyEPROM, EEPROM), or a flash memory, among others. The volatile memory may be random access memory (Random Access Memory, RAM), static random access memory (STATIC RAM, SRAM), dynamic random access memory (DYNAMIC RAM, DRAM), synchronous Dynamic Random Access Memory (SDRAM), double data rate Synchronous dynamic random access memory (Double DATA RATE SDRAM, DDRSDRAM), enhanced Synchronous dynamic random access memory (ENHANCED SDRAM, ESDRAM), synchronous link dynamic random access memory (SYNCHLINK DRAM, SLDRAM), and Direct random access memory (DRRAM). Memory 809 in embodiments of the application includes, but is not limited to, these and any other suitable types of memory.

The processor 810 may include one or more processing units; optionally, the processor 810 integrates an application processor that primarily processes operations involving an operating system, user interface, application programs, etc., and a modem processor that primarily processes wireless communication signals, such as a baseband processor. It will be appreciated that the modem processor described above may not be integrated into the processor 810.

The embodiment of the application also provides a readable storage medium, and the readable storage medium stores a program or an instruction, which when executed by a processor, implements each process of the above method embodiment, and can achieve the same technical effects, so that repetition is avoided, and no further description is provided herein.

The processor is a processor in the electronic device in the above embodiment. Readable storage media include computer readable storage media such as Read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), magnetic or optical disks, and the like.

The embodiment of the application further provides a chip, the chip comprises a processor and a communication interface, the communication interface is coupled with the processor, the processor is used for running programs or instructions, the processes of the embodiment of the method can be realized, the same technical effects can be achieved, and the repetition is avoided, and the description is omitted here.

It should be understood that the chips referred to in the embodiments of the present application may also be referred to as system-on-chip chips, chip systems, or system-on-chip chips, etc.

Embodiments of the present application provide a computer program product stored in a storage medium, where the program product is executed by at least one processor to implement the respective processes of the above method embodiments, and achieve the same technical effects, and for avoiding repetition, a detailed description is omitted herein.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element. Furthermore, it should be noted that the scope of the methods and apparatus in the embodiments of the present application is not limited to performing the functions in the order shown or discussed, but may also include performing the functions in a substantially simultaneous manner or in an opposite order depending on the functions involved, e.g., the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present application may be embodied essentially or in part in the form of a computer software product stored on a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising instructions for causing a terminal (which may be a mobile phone, a computer, a server, or a network device, etc.) to perform the method of the embodiments of the present application.

The embodiments of the present application have been described above with reference to the accompanying drawings, but the present application is not limited to the above-described embodiments, which are merely illustrative and not restrictive, and many forms may be made by those having ordinary skill in the art without departing from the spirit of the present application and the scope of the claims, which are to be protected by the present application.

Claims

1. A method of processing an image, the method comprising:

And generating a second image according to the first noise reduction image and the first edge image.

2. The processing method of claim 1, wherein prior to said inputting the first image and the second noise information into a first image processing network model, the method further comprises:

Acquiring a third image;

Adding noise data to the third image based on third noise information to obtain a noise image;

Performing edge detection on the third image to obtain a second edge image;

generating training data according to the third image, the third noise information, the noise image and the second edge image;

and training the first neural network model through the training data and the target loss function to obtain the first image processing network model.

3. The processing method according to claim 1, wherein before the second noise information is determined based on the first noise information and the noise adjustment weight value, the processing method further comprises:

Receiving noise reduction weight input by a user;

and determining the noise adjustment weight value according to the noise reduction weight.

4. The processing method according to claim 1, wherein before the second noise information is determined based on the first noise information and the noise adjustment weight value, the processing method further comprises:

Performing semantic segmentation processing on the first image to obtain the region type of the at least one image region;

the noise adjustment weight value is determined based on the region type.

5. The processing method of claim 1, wherein the first image processing network model comprises an encoder network;

Before said inputting said first image and said second noise information into the first image processing network model, said method further comprises:

Performing splicing processing on the first image and the second noise information to obtain an input vector;

The input vector is input to the encoder network.

6. An image processing apparatus, characterized in that the processing apparatus comprises:

A determining module configured to determine second noise information based on the first noise information and a noise adjustment weight value, wherein the noise adjustment weight value is used to indicate a noise adjustment strength of at least one image region of the first image;

The processing module is used for inputting the first image and the second noise information into a first image processing network model and acquiring a first noise reduction image and a first edge image which are generated by the first image processing network model according to the first image and the second noise information;

and the generation module is used for generating a second image according to the first noise reduction image and the first edge image.

7. The processing device of claim 6, wherein the processing device further comprises:

the acquisition module is used for acquiring a third image;

The processing module is further used for adding noise data to the third image based on third noise information to obtain a noise image;

the generating module is further configured to generate training data according to the third image, the third noise information, the noise image, and the second edge image;

and the training module is used for training the first neural network model through the training data and the target loss function to obtain the first image processing network model.

8. The processing device of claim 6, wherein the processing device further comprises:

The receiving module is used for receiving the noise reduction weight input by the user;

the determining module is further configured to determine the noise adjustment weight value according to the noise reduction weight.

9. The processing apparatus according to claim 6, wherein,

The processing module is further used for carrying out semantic segmentation processing on the first image to obtain the region type of the at least one image region;

The determining module is further configured to determine the noise adjustment weight value based on the region type.

10. The processing apparatus of claim 6, wherein the first image processing network model comprises an encoder network;

The processing module is further used for performing splicing processing on the first image and the second noise information to obtain an input vector; and

The input vector is input to the encoder network.