CN110213462B - Image processing method, image processing device, electronic apparatus, image processing circuit, and storage medium - Google Patents
Image processing method, image processing device, electronic apparatus, image processing circuit, and storage medium Download PDFInfo
- Publication number
- CN110213462B CN110213462B CN201910509584.8A CN201910509584A CN110213462B CN 110213462 B CN110213462 B CN 110213462B CN 201910509584 A CN201910509584 A CN 201910509584A CN 110213462 B CN110213462 B CN 110213462B
- Authority
- CN
- China
- Prior art keywords
- image
- noise
- image processing
- noise reduction
- processor
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012545 processing Methods 0.000 title claims abstract description 48
- 238000003672 processing method Methods 0.000 title claims abstract description 27
- 230000009467 reduction Effects 0.000 claims abstract description 74
- 238000013473 artificial intelligence Methods 0.000 claims abstract description 30
- 238000003062 neural network model Methods 0.000 claims description 24
- 238000004590 computer program Methods 0.000 claims description 6
- 238000013507 mapping Methods 0.000 claims description 5
- 238000012549 training Methods 0.000 claims description 4
- 230000002194 synthesizing effect Effects 0.000 claims description 3
- 230000015572 biosynthetic process Effects 0.000 claims 1
- 238000003786 synthesis reaction Methods 0.000 claims 1
- 238000000034 method Methods 0.000 abstract description 20
- 230000000694 effects Effects 0.000 abstract description 13
- 238000010586 diagram Methods 0.000 description 12
- 230000008569 process Effects 0.000 description 8
- 230000006870 function Effects 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000003384 imaging method Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000003705 background correction Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 230000007704 transition Effects 0.000 description 2
- 238000012896 Statistical algorithm Methods 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 239000000872 buffer Substances 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000012407 engineering method Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000002068 genetic effect Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 239000004984 smart glass Substances 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
- H04N23/81—Camera processing pipelines; Components thereof for suppressing or minimising disturbance in the image signal generation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20021—Dividing image into blocks, subimages or windows
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Studio Devices (AREA)
- Image Processing (AREA)
Abstract
本申请提出一种图像处理方法、装置、电子设备及图像处理电路，该方法包括获取原始图像；结合预设特征，对原始图像进行划分得到多个图像区域，各图像区域，基于预设特征的特征值不同；针对各图像区域分别基于人工智能降噪，以得到目标降噪图像。通过本申请能够精准地识别出图像中各个图像特征区域的噪声特性，实现结合图像特征进行针对性的降噪，提升降噪效果。
The present application provides an image processing method, device, electronic device and image processing circuit. The method includes acquiring an original image; combining with preset features, dividing the original image to obtain multiple image regions, and each image region has an image region based on the preset feature. The eigenvalues are different; each image area is denoised based on artificial intelligence to obtain the target denoised image. Through the present application, the noise characteristics of each image feature area in the image can be accurately identified, targeted noise reduction can be implemented in combination with the image characteristics, and the noise reduction effect can be improved.
Description
Technical Field
The present disclosure relates to the field of image processing technologies, and in particular, to an image processing method and apparatus, an electronic device, and an image processing circuit.
Background
In the field of image noise reduction, an artificial intelligence noise reduction model based on deep learning generally carries out image noise reduction, the artificial intelligence noise reduction model can carry out characteristic identification on noise in an image, and a corresponding noise reduction mode is adopted to reduce noise of the image according to the identified noise characteristic, namely, the same artificial intelligence noise reduction model is adopted to carry out noise reduction processing on the same image.
In this way, the noise characteristics of each image feature region in the image cannot be accurately identified, and noise reduction is not targeted and the noise reduction effect is not good.
Disclosure of Invention
The present application is directed to solving, at least to some extent, one of the technical problems in the related art.
Therefore, an object of the present application is to provide an image processing method, an image processing apparatus, an electronic device, and an image processing circuit, which can accurately identify noise characteristics of each image feature region in an image, implement targeted noise reduction in combination with image features, and improve noise reduction effect.
In order to achieve the above object, an embodiment of the first aspect of the present application provides an image processing method, including: acquiring an original image; dividing the original image to obtain a plurality of image areas by combining preset features, wherein the image areas are different in feature value based on the preset features; and denoising the image areas based on artificial intelligence respectively to obtain a target denoising image.
According to the image processing method provided by the embodiment of the first aspect of the application, the original image is divided to obtain the plurality of image areas by acquiring the original image and combining the preset features, the image areas are different based on the feature values of the preset features, and noise reduction is performed on the image areas respectively based on artificial intelligence to obtain the target noise reduction image, so that the noise characteristics of the image feature areas in the image can be accurately identified, targeted noise reduction is performed by combining the image features, and the noise reduction effect is improved.
In order to achieve the above object, an embodiment of a second aspect of the present application provides an image processing apparatus, including: the acquisition module is used for acquiring an original image; the dividing module is used for dividing the original image to obtain a plurality of image areas by combining preset features, wherein the image areas are different in feature value based on the preset features; and the noise reduction module is used for reducing noise of each image area based on artificial intelligence respectively so as to obtain a target noise reduction image.
According to the image processing device provided by the embodiment of the second aspect of the application, the original image is divided to obtain the plurality of image areas by acquiring the original image and combining the preset features, the image areas are different based on the feature values of the preset features, and noise reduction is performed on the image areas respectively based on artificial intelligence to obtain the target noise reduction image, so that the noise characteristics of the image feature areas in the image can be accurately identified, the targeted noise reduction is performed by combining the image features, and the noise reduction effect is improved.
In order to achieve the above object, an electronic device according to a third aspect of the present application includes: the image processing method comprises an image sensor, a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein the image sensor is electrically connected with the processor, and when the processor executes the program, the image processing method provided by the embodiment of the first aspect of the application is realized.
According to the electronic device provided by the embodiment of the third aspect of the application, the original image is divided to obtain the plurality of image areas by acquiring the original image and combining the preset features, the image areas are different based on the feature values of the preset features, and noise reduction is performed on the image areas respectively based on artificial intelligence to obtain the target noise reduction image, so that the noise characteristics of the image feature areas in the image can be accurately identified, the targeted noise reduction is performed by combining the image features, and the noise reduction effect is improved.
In order to achieve the above object, an embodiment of a fourth aspect of the present application provides an image processing circuit, including: the system comprises an image signal processing ISP processor and a graphic processor GPU; the ISP processor is electrically connected with the image sensor and is used for controlling the image sensor to acquire an original image; the GPU is electrically connected with the ISP processor and is used for dividing the original image to obtain a plurality of image areas by combining preset characteristics, and the characteristic values of the image areas are different based on the preset characteristics; and denoising the image areas based on artificial intelligence respectively to obtain a target denoising image.
According to the image processing circuit provided by the embodiment of the fourth aspect of the application, the original image is divided to obtain the plurality of image areas by acquiring the original image and combining the preset features, the image areas are different based on the feature values of the preset features, and noise reduction is performed on the image areas respectively based on artificial intelligence to obtain the target noise reduction image, so that the noise characteristics of the image feature areas in the image can be accurately identified, the targeted noise reduction is performed by combining the image features, and the noise reduction effect is improved.
To achieve the above object, a computer-readable storage medium is provided in an embodiment of the fifth aspect of the present application, on which a computer program is stored, which when executed by a processor implements the image processing method as provided in the embodiment of the first aspect of the present application.
According to the computer-readable storage medium provided by the embodiment of the fifth aspect of the application, the original image is divided by acquiring the original image and combining the preset features to obtain a plurality of image areas, the image areas are different based on the feature values of the preset features, and noise reduction is performed on the image areas based on artificial intelligence respectively to obtain a target noise reduction image, so that the noise characteristics of each image feature area in the image can be accurately identified, targeted noise reduction is performed by combining the image features, and the noise reduction effect is improved.
Additional aspects and advantages of the present application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the present application.
Drawings
The foregoing and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
fig. 1 is a schematic flowchart of an image processing method according to an embodiment of the present application;
FIG. 2 is a schematic diagram of image partitioning in an embodiment of the present application;
fig. 3 is a schematic flowchart of an image processing method according to another embodiment of the present application;
fig. 4 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of an image processing apparatus according to another embodiment of the present application;
fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure;
fig. 7 is a schematic diagram of an electronic device according to an embodiment of the present disclosure;
fig. 8 is a schematic diagram of an image processing circuit according to an embodiment of the present disclosure.
Detailed Description
Reference will now be made in detail to embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are exemplary only for the purpose of explaining the present application and are not to be construed as limiting the present application. On the contrary, the embodiments of the application include all changes, modifications and equivalents coming within the spirit and terms of the claims appended hereto.
In order to solve the technical problems that noise characteristics of each image feature region in an image cannot be accurately identified, noise reduction is not targeted, and noise reduction effect is poor in the related art, the embodiment of the application provides an image processing method.
Fig. 1 is a schematic flowchart of an image processing method according to an embodiment of the present application.
The image processing method based on the multi-frame image is applied to electronic equipment, and the electronic equipment can be hardware equipment with various operating systems and imaging equipment, such as a mobile phone, a tablet personal computer, a personal digital assistant and wearable equipment.
Referring to fig. 1, the method includes:
s101: an original image is acquired.
The RAW image may be, for example, an unprocessed RAW format image acquired by an image sensor of the electronic device, which is not limited in this respect.
The RAW format image is an original image obtained by converting a captured light source signal into a digital signal by an image sensor. RAW format images record RAW information of the digital camera sensor, and also record some metadata generated by camera shooting, such as setting of sensitivity, shutter speed, aperture value, white balance, and the like.
Whether the current shot scene belongs to a night scene or not can be determined by acquiring a preview image of the current shot scene. Because the environmental brightness values in different scenes are different, the contents of the preview images are also different, and the night scene shooting mode can be started to acquire the original image after the current shooting scene is determined to belong to the night scene according to the picture content of the preview image of the current shooting scene and the environmental brightness values of all areas.
For example, the picture content of the preview image includes a night sky, a night scene light source, or the like, or the environment brightness value in each region of the preview image conforms to the brightness distribution characteristic of the image in the night scene environment, so that it can be determined that the current shooting scene belongs to the night scene.
S102: and combining the preset features, dividing the original image to obtain a plurality of image areas, wherein the feature values of the image areas are different based on the preset features.
Alternatively, the preset feature may be, for example, an image feature such as image contrast, sensitivity, white balance, or the like.
And the preset features are image brightness, and the targeted noise reduction is realized by combining the image brightness, and the general noise characteristic difference of image areas with different brightness features is larger, so that the noise reduction effect can be obviously improved, and the user experience is improved.
Optionally, the feature value of the image region based on the preset feature is a brightness mean value of all pixel points in the image region, or alternatively, the brightness variance value of the pixel points in the image region calculated based on a common statistical algorithm may also be used.
In a specific implementation process, the luminance value of each pixel point in the original image may be counted, and the luminance value of each pixel point is compared with a preset first luminance threshold and a preset second luminance threshold, where the first luminance threshold is smaller than the second luminance threshold, and according to a result obtained by the comparison, the original image is divided into a bright area, a bright-dark transition area, and a dark area, see fig. 2, where fig. 2 is an image division schematic diagram in the embodiment of the present application, the luminance values of all pixel points in the bright area are greater than or equal to the second luminance threshold, the luminance values of all pixel points in the bright-dark transition area are less than the second luminance threshold and greater than or equal to the first luminance threshold, and the luminance values of all pixel points in the dark area are less than the first luminance threshold.
In other embodiments, the image may be further divided into a plurality of image regions based on some other image division algorithms, and each image region is different based on a feature value of the preset feature, which is not limited herein.
S103: and denoising each image region based on artificial intelligence respectively to obtain a target denoising image.
Because the image sensor in the electronic device is subjected to different degrees of photo-electromagnetic interference between peripheral circuits and pixels of the image sensor in the electronic device during shooting, noise inevitably exists in the shot original image, and the definition of the shot image is different due to different interference degrees. Therefore, the acquired original image also has noise, and artificial intelligence noise reduction can be further performed on each image area to obtain a target noise reduction image.
Optionally, in some embodiments, referring to fig. 3, the noise reduction based on artificial intelligence for each image region includes:
s301: carrying out noise characteristic identification on each image area by adopting a neural network model and combining the characteristic values corresponding to each image area; the neural network model learns the mapping relation between the characteristic value and the noise characteristic of each image area.
Optionally, the neural network model is trained by using the sample images of the feature values until the noise characteristics identified by the neural network model are matched with the noise characteristics labeled in the corresponding sample images, and the training of the neural network model is completed.
As a possible implementation, since the neural network model has learned the mapping relationship between the feature value and the noise characteristic of each image region. Therefore, each image area can be respectively input into the neural network model, so that the noise characteristics of each image area are respectively identified by adopting the neural network model, the noise characteristics corresponding to each image area are identified, the aim of targeted noise reduction is fulfilled, the noise reduction effect is improved, and the noise reduction power consumption is reduced to the maximum extent.
Of course, the neural network model is only one possible implementation manner for implementing noise reduction based on artificial intelligence, and in the actual implementation process, noise reduction based on artificial intelligence may be implemented in any other possible manner, for example, it may also be implemented by using a conventional programming technique (simulation method and engineering method), or for example, it may also be implemented by using a genetic algorithm and an artificial neural network method.
S302: and denoising each image region according to the identified noise characteristics.
In the embodiment of the present application, the noise characteristic may be a statistical characteristic of random noise caused by the image sensor. The noise mainly includes thermal noise and shot noise, where the thermal noise conforms to a gaussian distribution, and the shot noise conforms to a poisson distribution, and the statistical characteristic in the embodiment of the present application may refer to a variance value of the noise, and may also be a value of other possible situations, which is not limited herein.
In the embodiment of the application, after denoising each image region according to the identified noise characteristics, the image region obtained by denoising may be further used as a target denoising region, and each target denoising region is synthesized to obtain a target denoising image.
The regions obtained by denoising each image region may be referred to as target denoising regions, and the number of the target denoising regions is the same as the number of the image regions obtained by dividing the original image.
In the specific implementation process, the target noise reduction regions may be synthesized based on a stitching algorithm to obtain a target noise reduction image, which is not limited.
In the embodiment, the original image is obtained and combined with the preset features, the original image is divided to obtain a plurality of image areas, each image area is different in feature value based on the preset features, and noise reduction is performed on each image area based on artificial intelligence to obtain a target noise reduction image, so that the noise characteristics of each image feature area in the image can be accurately identified, targeted noise reduction is performed by combining the image features, and the noise reduction effect is improved.
Fig. 4 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present application.
Referring to fig. 4, the apparatus 400 includes:
an obtaining module 401, configured to obtain an original image;
a dividing module 402, configured to divide an original image to obtain a plurality of image areas in combination with preset features, where feature values of the image areas are different based on the preset features;
and a noise reduction module 403, configured to perform noise reduction on each image region based on artificial intelligence, respectively, to obtain a target noise-reduced image.
Optionally, in some embodiments, the noise reduction module 403 is specifically configured to:
carrying out noise characteristic identification on each image area by adopting a neural network model and combining the characteristic values corresponding to each image area; the neural network model learns the mapping relation between the characteristic value and the noise characteristic of each image area;
and denoising each image region according to the identified noise characteristics.
Optionally, in some embodiments, the neural network model is trained by using the sample images of each feature value until the training of the neural network model is completed when the noise characteristics identified by the neural network model match the noise characteristics labeled in the corresponding sample images.
Optionally, in some embodiments, referring to fig. 5, further comprising:
and a synthesizing module 404, configured to take the image region obtained by noise reduction as a target noise reduction region, and synthesize each target noise reduction region to obtain a target noise reduction image.
Optionally, in some embodiments, the preset feature is image brightness.
Optionally, in some embodiments, the feature value of the image region based on the preset feature is a luminance average value of all pixel points in the image region.
It should be noted that the foregoing explanation of the embodiment of the image processing method is also applicable to the image processing apparatus 400 of this embodiment, and is not repeated here.
In the embodiment, the original image is obtained and combined with the preset features, the original image is divided to obtain a plurality of image areas, each image area is different in feature value based on the preset features, and noise reduction is performed on each image area based on artificial intelligence to obtain a target noise reduction image, so that the noise characteristics of each image feature area in the image can be accurately identified, targeted noise reduction is performed by combining the image features, and the noise reduction effect is improved.
In order to implement the foregoing embodiment, the present application further provides an electronic device 200, referring to fig. 6, where fig. 6 is a schematic structural diagram of an electronic device provided in an embodiment of the present application, and the electronic device includes: the image processing system comprises an image sensor 210, a memory 230, a processor 220 and a computer program which is stored on the memory 230 and can run on the processor 220, wherein the image sensor 210 is electrically connected with the processor 220, and the image processing method is realized when the processor 220 executes the program.
As one possible scenario, the processor 220 may include: an image signal processing ISP processor.
The ISP processor is used for controlling the image sensor to acquire an original image.
As another possible scenario, the processor 220 may further include: and a Graphics Processing Unit (GPU) connected to the ISP processor.
The GPU is used for dividing an original image to obtain a plurality of image areas by combining preset characteristics, and the characteristic values of the image areas are different based on the preset characteristics; and denoising each image region based on artificial intelligence respectively to obtain a target denoising image.
Referring to fig. 7 as an example, on the basis of the electronic device in fig. 6, fig. 7 is a schematic diagram illustrating an electronic device according to an embodiment of the present application. The memory 230 of the electronic device 200 includes the non-volatile memory 60, the internal memory 62, and the processor 220. Memory 230 has stored therein computer readable instructions. The computer readable instructions, when executed by the memory, cause the processor 220 to perform the image processing method of any of the above embodiments.
As shown in fig. 7, the electronic apparatus 200 includes a processor 220, a nonvolatile memory 60, an internal memory 62, a display screen 63, and an input device 64 connected via a system bus 61. The non-volatile memory 60 of the electronic device 200 stores, among other things, an operating system and computer readable instructions. The computer readable instructions can be executed by the processor 220 to implement the image processing method of the embodiment of the present application. The processor 220 is used to provide computing and control capabilities that support the operation of the overall electronic device 200. The internal memory 62 of the electronic device 200 provides an environment for the execution of computer-readable instructions in the non-volatile memory 60. The display 63 of the electronic device 200 may be a liquid crystal display or an electronic ink display, and the input device 64 may be a touch layer covered on the display 63, a button, a trackball or a touch pad arranged on a housing of the electronic device 200, or an external keyboard, a touch pad or a mouse. The electronic device 200 may be a mobile phone, a tablet computer, a notebook computer, a personal digital assistant, or a wearable device (e.g., a smart bracelet, a smart watch, a smart helmet, smart glasses), etc.
It will be understood by those skilled in the art that the structure shown in fig. 7 is only a schematic diagram of a part of the structure related to the present application, and does not constitute a limitation to the electronic device 200 to which the present application is applied, and a specific electronic device 200 may include more or less components than those shown in the drawings, or combine some components, or have a different arrangement of components.
To implement the foregoing embodiments, an image processing circuit is further provided in the present application, please refer to fig. 8, fig. 8 is a schematic diagram of an image processing circuit according to an embodiment of the present application, and as shown in fig. 8, an image processing circuit 70 includes an image signal processing ISP processor 71 (the ISP processor 71 is used as the processor 220) and a graphics processor GPU.
The ISP processor is electrically connected with the image sensor and is used for controlling the image sensor to acquire an original image;
the GPU is electrically connected with the ISP processor and is used for dividing the original image to obtain a plurality of image areas by combining preset characteristics, and the characteristic values of all the image areas are different based on the preset characteristics; and denoising each image region based on artificial intelligence respectively to obtain a target denoising image.
The image data captured by the camera 73 is first processed by the ISP processor 71, and the ISP processor 71 analyzes the image data to capture image statistics that may be used to determine one or more control parameters of the camera 73. The camera module 310 may include one or more lenses 732 and an image sensor 734. The image sensor 734 may include an array of color filters (e.g., Bayer filters), and the image sensor 734 may acquire the light intensity and wavelength information captured by each imaging pixel and provide a raw set of image data that may be processed by the ISP processor 71. The sensor 74 (e.g., gyroscope) may provide parameters of the acquired image processing (e.g., anti-shake parameters) to the ISP processor 71 based on the type of interface of the sensor 74. The sensor 74 interface may be a SMIA (Standard Mobile Imaging Architecture) interface, other serial or parallel camera interface, or a combination of the above.
Further, the image sensor 734 may also send raw image data to the sensor 74, the sensor 74 may provide the raw image data to the ISP processor 71 based on the sensor 74 interface type, or the sensor 74 may store the raw image data in the image memory 75.
The ISP processor 71 processes the raw image data pixel by pixel in a variety of formats. For example, each image pixel may have a bit depth of 8, 10, 12, or 14 bits, and ISP processor 71 may perform one or more image processing operations on the raw image data, gathering statistical information about the image data. Wherein the image processing operations may be performed with the same or different bit depth precision.
The ISP processor 71 may also receive image data from an image memory 75. For example, the sensor 74 interface sends raw image data to the image memory 75, and the raw image data in the image memory 75 is then provided to the ISP processor 71 for processing. The image Memory 75 may be the Memory 330, a portion of the Memory 330, a storage device, or a separate dedicated Memory within the electronic device, and may include a DMA (Direct Memory Access) feature.
Upon receiving raw image data from image sensor 734 interface or from sensor 74 interface or from image memory 75, ISP processor 71 may perform one or more image processing operations, such as temporal filtering. The processed image data may be sent to image memory 75 for additional processing before being displayed. The ISP processor 71 receives the processed data from the image memory 75 and performs image data processing on the processed data in the raw domain and in the RGB and YCbCr color spaces. The image data processed by ISP processor 71 may be output to display 77 (display 77 may include display screen 63) for viewing by a user and/or further processing by a graphics engine or GPU. Further, the output of the ISP processor 71 may also be sent to an image memory 75, and the display 77 may read image data from the image memory 75. In one embodiment, image memory 75 may be configured to implement one or more frame buffers. In addition, the output of the ISP processor 71 may be sent to an encoder/decoder 76 for encoding/decoding the image data. The encoded image data may be saved and decompressed before being displayed on the display 77 device. The encoder/decoder 76 may be implemented by a CPU or GPU or coprocessor.
The statistical data determined by the ISP processor 71 may be sent to the control logic 72 unit. For example, the statistical data may include image sensor 734 statistical information such as auto-exposure, auto-white balance, auto-focus, flicker detection, black level compensation, lens 732 shading correction, and the like. Control logic 72 may include a processing element and/or microcontroller that executes one or more routines (e.g., firmware) that determine control parameters for camera 73 and control parameters for ISP processor 71 based on the received statistical data. For example, the control parameters of camera 73 may include sensor 74 control parameters (e.g., gain, integration time for exposure control, anti-shake parameters, etc.), camera flash control parameters, lens 732 control parameters (e.g., focal length for focusing or zooming), or a combination of these parameters. The ISP control parameters may include gain levels and color correction matrices for automatic white balance and color adjustment (e.g., during RGB processing), and lens 732 shading correction parameters.
The following steps are used for realizing the image processing method by using the image processing technology in the figure 8: the ISP processor controls the image sensor to acquire an original image; the GPU combines preset characteristics to divide an original image to obtain a plurality of image areas, and the characteristic values of the image areas are different based on the preset characteristics; and denoising each image region based on artificial intelligence respectively to obtain a target denoising image.
To implement the foregoing embodiments, the present application further provides a storage medium, and when instructions in the storage medium are executed by a processor, the processor executes the following steps: acquiring an original image; dividing an original image to obtain a plurality of image areas by combining preset characteristics, wherein the characteristic values of the image areas are different based on the preset characteristics; and denoising each image region based on artificial intelligence respectively to obtain a target denoising image.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware related to instructions of a computer program, and the program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), or the like.
The above examples only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
It should be noted that, in the description of the present application, the terms "first", "second", etc. are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. In addition, in the description of the present application, "a plurality" means two or more unless otherwise specified.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and the scope of the preferred embodiments of the present application includes other implementations in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present application.
It should be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.
In addition, functional units in the embodiments of the present application may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium.
The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc.
In the description herein, reference to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
Although embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application, and that variations, modifications, substitutions and alterations may be made to the above embodiments by those of ordinary skill in the art within the scope of the present application.
Claims (15)
1. An image processing method, comprising:
acquiring an original image;
dividing the original image to obtain a plurality of image areas by combining preset characteristics of all pixel points in the original image, wherein the preset characteristics are the brightness of the original image based on different characteristic values of the preset characteristics, and the characteristic values of the preset characteristics are the brightness average values of all the pixel points in the image areas;
respectively denoising the image regions based on artificial intelligence to obtain a target denoising image, wherein the denoising based on artificial intelligence for each image region comprises:
performing noise characteristic identification on each image area by adopting a neural network model and combining the characteristic values corresponding to each image area; the neural network model learns the mapping relation between the characteristic value of each image area and the noise characteristic, and the noise characteristic refers to the variance value of noise;
and denoising each image region according to the identified noise characteristics.
2. The image processing method according to claim 1, wherein the neural network model is trained by using the sample images of the feature values until the noise characteristics identified by the neural network model match the noise characteristics labeled in the corresponding sample images, and the training of the neural network model is completed.
3. The image processing method according to claim 1, further comprising:
taking the image area obtained by noise reduction as a target noise reduction area;
and synthesizing the target noise reduction areas to obtain the target noise reduction image.
4. The image processing method according to any one of claims 1 to 3, wherein the preset feature is image brightness.
5. The image processing method according to claim 4, wherein the feature value of the image region based on the predetermined feature is a luminance average of all pixels in the image region.
6. An image processing apparatus characterized by comprising:
the acquisition module is used for acquiring an original image;
the dividing module is used for dividing the original image to obtain a plurality of image areas by combining preset characteristics of all pixel points in the original image, wherein the preset characteristics are the brightness of the original image based on different characteristic values of the preset characteristics, and the characteristic values of the preset characteristics are the brightness average values of all the pixel points in the image areas;
the noise reduction module is used for reducing noise based on artificial intelligence aiming at each image area to obtain a target noise reduction image, and reducing noise based on artificial intelligence aiming at each image area, and comprises:
performing noise characteristic identification on each image area by adopting a neural network model and combining the characteristic values corresponding to each image area; the neural network model learns the mapping relation between the characteristic value of each image area and the noise characteristic, and the noise characteristic refers to the variance value of noise;
and denoising each image region according to the identified noise characteristics.
7. The image processing apparatus according to claim 6, wherein the neural network model is trained by using the sample images of the feature values until the noise characteristics identified by the neural network model match the noise characteristics labeled in the corresponding sample images, and the training of the neural network model is completed.
8. The image processing apparatus according to claim 6, further comprising:
and the synthesis module is used for taking the image area obtained by noise reduction as a target noise reduction area and synthesizing each target noise reduction area to obtain the target noise reduction image.
9. The image processing apparatus according to any one of claims 6 to 8, wherein the preset feature is an image brightness.
10. The image processing apparatus according to claim 9, wherein the feature value of the image region based on the predetermined feature is a luminance average of all pixels in the image region.
11. An electronic device, comprising: an image sensor, a memory, a processor and a computer program stored on the memory and executable on the processor, the image sensor being electrically connected to the processor, when executing the program, implementing the image processing method according to any of claims 1-5.
12. The electronic device of claim 11, wherein the processor comprises an Image Signal Processing (ISP) processor;
and the ISP processor is used for controlling the image sensor to acquire an original image.
13. The electronic device of claim 12, wherein the processor comprises a Graphics Processor (GPU) coupled to the ISP processor;
the GPU is used for dividing the original image to obtain a plurality of image areas by combining preset characteristics, and the characteristic values of the image areas are different based on the preset characteristics; and denoising the image areas based on artificial intelligence respectively to obtain a target denoising image.
14. An image processing circuit, characterized in that the image processing circuit comprises an image signal processing ISP processor and a graphics processor GPU;
the ISP processor is electrically connected with the image sensor and is used for controlling the image sensor to acquire an original image;
the GPU is electrically connected with the ISP processor and is used for dividing the original image to obtain a plurality of image areas by combining preset characteristics, and the characteristic values of the image areas are different based on the preset characteristics; and denoising the image areas based on artificial intelligence respectively to obtain a target denoising image.
15. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the image processing method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910509584.8A CN110213462B (en) | 2019-06-13 | 2019-06-13 | Image processing method, image processing device, electronic apparatus, image processing circuit, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910509584.8A CN110213462B (en) | 2019-06-13 | 2019-06-13 | Image processing method, image processing device, electronic apparatus, image processing circuit, and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110213462A CN110213462A (en) | 2019-09-06 |
| CN110213462B true CN110213462B (en) | 2022-01-04 |
Family
ID=67792317
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910509584.8A Active CN110213462B (en) | 2019-06-13 | 2019-06-13 | Image processing method, image processing device, electronic apparatus, image processing circuit, and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110213462B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111131716B (en) * | 2019-12-31 | 2021-06-15 | 联想(北京)有限公司 | Image processing method and electronic device |
| CN113628124B (en) * | 2020-05-08 | 2024-01-16 | 深圳清华大学研究院 | ISP and visual task joint optimization method, system, medium and electronic equipment |
| JP7479959B2 (en) * | 2020-06-25 | 2024-05-09 | 富士フイルムヘルスケア株式会社 | Image processing device, medical imaging device, and image processing program |
| CN112613373A (en) * | 2020-12-16 | 2021-04-06 | 北京迈格威科技有限公司 | Image recognition method and device, electronic equipment and computer readable storage medium |
| CN114862685A (en) * | 2021-01-19 | 2022-08-05 | 杭州海康威视数字技术股份有限公司 | Image noise reduction method and image noise reduction module |
| CN113763275B (en) * | 2021-09-09 | 2024-07-12 | 深圳市文立科技有限公司 | Adaptive image noise reduction method, system and readable storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103973990A (en) * | 2014-05-05 | 2014-08-06 | 浙江宇视科技有限公司 | Wide dynamic fusion method and device |
| CN104134191A (en) * | 2014-07-11 | 2014-11-05 | 三星电子（中国）研发中心 | Image denoising method and image denoising device |
| CN105005973A (en) * | 2015-06-30 | 2015-10-28 | 广东欧珀移动通信有限公司 | Fast image denoising method and apparatus |
| CN106127729A (en) * | 2016-06-08 | 2016-11-16 | 浙江传媒学院 | A kind of picture noise level estimation method based on gradient |
| CN108876737A (en) * | 2018-06-06 | 2018-11-23 | 武汉大学 | A kind of image de-noising method of joint residual error study and structural similarity |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106331433B (en) * | 2016-08-25 | 2020-04-24 | 上海交通大学 | Video denoising method based on deep recurrent neural network |
| US10558750B2 (en) * | 2016-11-18 | 2020-02-11 | Salesforce.Com, Inc. | Spatial attention model for image captioning |
| CN108122206A (en) * | 2016-11-29 | 2018-06-05 | 深圳市中兴微电子技术有限公司 | A kind of low-light (level) image denoising method and device |
| US20200340897A1 (en) * | 2017-10-26 | 2020-10-29 | Essenlix Corporation | Devices and methods for monitoring liquid-solid contact time |
| CN109658344B (en) * | 2018-11-12 | 2022-10-25 | 哈尔滨工业大学（深圳） | Image denoising method, device, device and storage medium based on deep learning |
| CN109544477A (en) * | 2018-11-23 | 2019-03-29 | 南通大学 | Image denoising algorithm based on self-adapting dictionary study rarefaction representation |
| CN109859147B (en) * | 2019-03-01 | 2021-05-04 | 武汉大学 | A Real Image Denoising Method Based on Noise Modeling in Generative Adversarial Networks |
-
2019
- 2019-06-13 CN CN201910509584.8A patent/CN110213462B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103973990A (en) * | 2014-05-05 | 2014-08-06 | 浙江宇视科技有限公司 | Wide dynamic fusion method and device |
| CN104134191A (en) * | 2014-07-11 | 2014-11-05 | 三星电子（中国）研发中心 | Image denoising method and image denoising device |
| CN105005973A (en) * | 2015-06-30 | 2015-10-28 | 广东欧珀移动通信有限公司 | Fast image denoising method and apparatus |
| CN106127729A (en) * | 2016-06-08 | 2016-11-16 | 浙江传媒学院 | A kind of picture noise level estimation method based on gradient |
| CN108876737A (en) * | 2018-06-06 | 2018-11-23 | 武汉大学 | A kind of image de-noising method of joint residual error study and structural similarity |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110213462A (en) | 2019-09-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110062160B (en) | Image processing method and device | |
| CN109005364B (en) | Imaging control method, imaging control device, electronic device, and computer-readable storage medium | |
| CN110213462B (en) | Image processing method, image processing device, electronic apparatus, image processing circuit, and storage medium | |
| US11228720B2 (en) | Method for imaging controlling, electronic device, and non-transitory computer-readable storage medium | |
| CN108322669B (en) | Image acquisition method and apparatus, imaging apparatus, and readable storage medium | |
| CN110290289B (en) | Image noise reduction method, device, electronic device and storage medium | |
| CN110072052B (en) | Image processing method and device based on multi-frame image and electronic equipment | |
| CN110248106B (en) | Image noise reduction method, device, electronic device and storage medium | |
| CN109672819B (en) | Image processing method, apparatus, electronic device, and computer-readable storage medium | |
| CN109788207B (en) | Image synthesis method and device, electronic equipment and readable storage medium | |
| US20200045219A1 (en) | Control method, control apparatus, imaging device, and electronic device | |
| CN110191291B (en) | Image processing method and device based on multi-frame images | |
| CN110166708B (en) | Night scene image processing method and device, electronic equipment and storage medium | |
| CN109068058B (en) | Shooting control method, device and electronic device in super night scene mode | |
| CN109089046B (en) | Image noise reduction method and device, computer readable storage medium and electronic equipment | |
| CN110072051A (en) | Image processing method and device based on multiple image | |
| US11490024B2 (en) | Method for imaging controlling, electronic device, and non-transitory computer-readable storage medium | |
| CN108683862A (en) | Imaging control method, device, electronic device, and computer-readable storage medium | |
| WO2020207261A1 (en) | Image processing method and apparatus based on multiple frames of images, and electronic device | |
| CN107509044B (en) | Image synthesis method, image synthesis device, computer-readable storage medium and computer equipment | |
| CN108833802B (en) | Exposure control method and device and electronic equipment | |
| CN109040607B (en) | Imaging control method, imaging control device, electronic device and computer-readable storage medium | |
| CN110264420B (en) | Image processing method and device based on multi-frame images | |
| CN110166709B (en) | Night scene image processing method and device, electronic equipment and storage medium | |
| WO2020029679A1 (en) | Control method and apparatus, imaging device, electronic device and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |