CN113744138B - Image processing method, image processing device and storage medium - Google Patents

Abstract

The present disclosure relates to an image processing method, an image processing apparatus, and a storage medium. The image processing method comprises the steps of obtaining a single-channel gray level image, interpolating green channel pixels of the single-channel gray level image through an interpolation algorithm to obtain green channel horizontal direction pixel interpolation and green channel vertical direction pixel interpolation of the single-channel gray level image, carrying out self-adaptive interpolation fusion on the green channel horizontal direction pixel interpolation and the green channel vertical direction pixel interpolation by utilizing a pre-trained neural network model to obtain the green channel image of the single-channel gray level image, and carrying out complementation on red channel pixels and blue channel pixels of the single-channel gray level image by utilizing the interpolation algorithm by taking the green channel image as a reference template to obtain a color image. By the method, after the image is demosaiced, the color image with better image details and fewer pseudo colors is output.

Description

Image processing method, image processing apparatus, and storage medium

Technical Field

The present disclosure relates to the field of image processing technologies, and in particular, to an image processing method, an image processing apparatus, and a storage medium.

Background

The demosaicing technology is also called demosaick algorithm, and by complementing the missing pixels in the three channels of red (R), green (G) and blue (B), the single-channel gray level image can be expanded from the single-channel gray level image into a three-channel color image, and according to a certain strategy, the definition of the single-channel gray level image is enhanced in a self-adaptive manner, and the image noise is weakened.

With the development of image capturing technology, users have increasingly demanded to present images after photographing by an image capturing device. Furthermore, how to demosaict the single-channel gray level image obtained after the image is shot, the output image detail is better, and the pseudo color is less, is a new problem facing the demosaicing at present.

Disclosure of Invention

To overcome the problems in the related art, the present disclosure provides an image processing method, an image processing apparatus, and a storage medium.

According to a first aspect of the embodiment of the present disclosure, an image processing method is provided, the image processing method includes obtaining a single-channel gray image, interpolating green channel pixels of the single-channel gray image through an interpolation algorithm to obtain green channel horizontal direction pixel interpolation and green channel vertical direction pixel interpolation of the single-channel gray image, performing adaptive interpolation fusion on the green channel horizontal direction pixel interpolation and the green channel vertical direction pixel interpolation by using a pre-trained neural network model to obtain a green channel image of the single-channel gray image, and complementing red channel pixels and blue channel pixels of the single-channel gray image by using the interpolation algorithm with the green channel image as a reference template to obtain a color image.

In an example, the image processing method further comprises the steps of determining an image training set, wherein the image training set comprises color training images, preprocessing the color training images in the image training set to obtain preprocessed color training images, determining the preprocessed color training images as comparison images of the output images of the neural network model, performing mosaic processing on the comparison images to obtain single-channel gray training images corresponding to the comparison images, and training the neural network model based on the single-channel gray training images and the comparison images.

In one example, the image training set includes one or more of a high definition image, a high frequency texture image, an image captured by a designated camera module, and an image captured by a single lens reflex camera acquired based on a common high definition image set.

In one example, the contrast map is obtained after brightness and noise removal and edge enhancement processing.

In one example, the pre-trained neural network model is a U-shaped neural network model.

According to a second aspect of the embodiment of the present disclosure, there is provided an image processing apparatus including an acquisition unit configured to acquire a single-channel gray-scale image, a processing unit configured to interpolate green-channel pixels of the single-channel gray-scale image by an interpolation algorithm to obtain green-channel horizontal-direction pixel interpolation and green-channel vertical-direction pixel interpolation of the single-channel gray-scale image, and adaptively interpolate and fuse the green-channel horizontal-direction pixel interpolation and the green-channel vertical-direction pixel interpolation by using a pre-trained neural network model to obtain a green-channel image of the single-channel gray-scale image, and complement red-channel pixels and blue-channel pixels of the single-channel gray-scale image by using the interpolation algorithm as a reference template to obtain a color image.

In an example, the image processing device further comprises a determining unit configured to determine an image training set, wherein the image training set comprises color training images, the processing unit is further configured to pre-process the color training images in the image training set to obtain pre-processed color training images, determine the pre-processed color training images as comparison graphs of the neural network model output images, perform mosaic processing on the comparison graphs to obtain single-channel gray training images corresponding to the comparison graphs, and the training unit is configured to train the neural network model based on the single-channel gray training images and the comparison graphs.

In one example, the image training set includes one or more of a high definition image, a high frequency texture image, an image captured by a designated camera module, and an image captured by a single lens reflex camera acquired based on a common high definition image set.

In one example, the contrast map is obtained after brightness and noise removal and edge enhancement processing.

In one example, the pre-trained neural network model is a U-shaped neural network model.

According to a third aspect of the present disclosure, there is provided an image processing apparatus including a memory configured to store instructions. And a processor configured to invoke instructions to perform the image processing method of the foregoing first aspect or any of the examples of the first aspect.

According to a fourth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer-executable instructions which, when executed by a processor, perform the image processing method of the first aspect or any of the examples of the first aspect.

The technical scheme provided by the embodiment of the disclosure can have the following beneficial effects that before interpolation of the pixels in the horizontal direction and interpolation and splicing of the pixels in the vertical direction of the green channel of the single-channel gray image, the interpolation of the pixels in the horizontal direction and the interpolation of the pixels in the vertical direction of the green channel of the single-channel gray image are corrected and fused by utilizing the self-adaptive interpolation characteristic of a pre-trained neural network model, so that a green channel image with high accuracy can be obtained, and further, a color image with accurate color is output by utilizing an interpolation algorithm according to the green channel image with high accuracy. By the method and the device, after the image is demosaiced, the color image with better image details and fewer pseudo colors can be output.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure.

Fig. 1 is a flowchart illustrating an image processing method according to an exemplary embodiment.

Fig. 2 is a diagram illustrating a Unet network architecture employing the present disclosure, according to one example embodiment.

FIG. 3 is a flowchart of a training Unet model, according to an example embodiment.

Fig. 4 is a block diagram of an image processing apparatus according to an exemplary embodiment.

Fig. 5 is a block diagram of an apparatus according to an example embodiment.

Detailed Description

Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as detailed in the accompanying claims.

The technical scheme of the exemplary embodiment of the present disclosure may be applied to an application scene photographed using a terminal including a photographing function. In the exemplary embodiments described below, the terminal is sometimes also referred to as an intelligent terminal device, where the terminal may be a Mobile terminal, and may also be referred to as a User Equipment (UE), a Mobile Station (MS), or the like. A terminal is a device that provides a user with a voice and/or data connection, or a chip provided in the device, for example, a handheld device having a wireless connection function, an in-vehicle device, or the like. Examples of terminals may include, for example, cell phones, tablet computers, notebook computers, palm computers, mobile internet devices (Mobile INTERNET DEVICES, MID), wearable devices, virtual Reality (VR) devices, augmented Reality (Augmented Reality, AR) devices, wireless terminals in industrial control, wireless terminals in unmanned driving, wireless terminals in teleoperation, wireless terminals in smart grids, wireless terminals in transportation security, wireless terminals in smart cities, wireless terminals in smart homes, and the like.

Currently, the main stream demosaick algorithm implements demosaick using interpolation algorithms. The core idea is to complement R, B channels by predicting the difference between R, B channels and G channels based on the interpolation result of the G channels. However, with the development of the image capturing technology, how to demosaicing better, outputting images with better details and fewer pseudo colors is a new problem facing the current demosaicing.

Furthermore, to further improve the effect achieved by implementing demosaick using interpolation algorithms, the most popular deep convolutional neural network in artificial intelligence (ARTIFICIAL INTELLIGENCE, AI) is beginning to be used in demosaick. However, the convolutional neural network can improve the demosaicing performance, and has the advantages of complex structure, excessive calling parameters and low running speed. This is the biggest problem of AI-demosaick at present.

Based on this, the present disclosure provides an image processing method that can demosaict an image in combination with an interpolation algorithm and a convolutional neural network. On the premise of ensuring that the image demosaicing processing speed is relatively high and meeting the commercial landing requirement, the demosaicing processing performance of the image is improved.

Fig. 1 is a flowchart illustrating an image processing method according to an exemplary embodiment, and as shown in fig. 1, the image processing method includes the following steps.

In step S11, a single-channel gray-scale image is acquired.

In the present disclosure, the single-channel gray-scale image may be a single-channel gray-scale image captured by an image sensor in an image capturing apparatus when captured with the image capturing apparatus. The single-channel gray image may be, for example, a single-channel gray image of a bayer array arrangement.

The image pickup device may be a digital camera, a single-lens reflex camera, an image pickup device included in a smart phone, or the like.

In step S12, the interpolation algorithm is used to interpolate the green channel pixels of the single-channel gray-scale image, so as to obtain the green channel horizontal direction pixel interpolation and the green channel vertical direction pixel interpolation of the single-channel gray-scale image.

In the present disclosure, the interpolation algorithm may be, for example, a bilinear interpolation algorithm, a bicubic interpolation algorithm, a residual class (ResidualInterpolation, RI) interpolation algorithm, or the like. The residual class interpolation algorithm may be, for example, a minimum Laplace residual interpolation algorithm (Minimized-LAPLACIAN RESIDUAL INTERPOLATION, MLRI) or an adaptive residual interpolation (AdaptiveResidual Interpolation, ARI).

In the present disclosure, for example, the green channel pixels of the single channel gray scale image may be interpolated as follows:

The single channel gray scale image is split into an R channel pixel image, a G channel pixel image, and a B channel pixel image. And (3) carrying out green interpolation correction (Green Interpolation Correction, GIC) on the G-channel pixel image of the single-channel gray level image, namely correcting errors of the G-channel pixel image between the green and red color components and errors of the G-channel pixel image between the green and blue color components, and then interpolating the green-channel pixel corrected by the GIC by utilizing an interpolation algorithm to obtain green-channel horizontal-direction pixel interpolation and green-channel vertical-direction pixel interpolation of the single-channel gray level image.

In step S13, the interpolation of the pixels in the horizontal direction and the interpolation of the pixels in the vertical direction of the green channel of the single-channel gray-scale image are adaptively interpolated and fused by using the pre-trained neural network model, so as to obtain the green channel image of the single-channel gray-scale image.

At present, when an interpolation algorithm is used for demosaicing an image, the obtained pixel interpolation in the horizontal direction and the pixel interpolation in the vertical direction of the G channel can be directly spliced to obtain a green channel image.

Because the interpolation algorithm is used for interpolating the G channel pixels, the obtained G channel horizontal direction pixel interpolation and the obtained vertical direction pixel interpolation have errors, and the obtained G channel horizontal direction pixel interpolation and the obtained vertical direction pixel interpolation are directly spliced, the color of the obtained green channel image is inaccurate. And then taking the green channel image with inaccurate color as a reference template, and when the interpolation algorithm is used for supplementing the red channel pixels and the blue channel pixels of the single-channel gray level image, the color of the obtained red channel image and the color of the blue channel image are inaccurate.

Therefore, the method and the device can correct the interpolation of the pixels in the horizontal direction and the interpolation of the pixels in the vertical direction of the green channel of the single-channel gray image by utilizing the self-adaptive interpolation characteristic of the pre-trained neural network model before the interpolation of the pixels in the horizontal direction and the interpolation of the pixels in the vertical direction of the green channel of the single-channel gray image are spliced, and then fuse the corrected interpolation of the pixels in the horizontal direction and the corrected interpolation of the pixels in the vertical direction of the green channel, so that the green channel image with high accuracy is obtained.

The pre-trained neural network model may be, for example, a U-shaped neural network model (Unet) or a demosaiced convolutional neural network (Demosaicking Convolutional Neural Network, dmCNN).

In the present disclosure, a pre-trained neural network model is taken as an example Unet, and the interpolation of pixels in the horizontal direction and the interpolation of pixels in the vertical direction of a green channel of a single-channel gray-scale image are adaptively interpolated and fused by using the pre-trained neural network model to obtain the green channel image of the single-channel gray-scale image.

Because Unet model can carry out self-adaptive correction to the horizontal pixel channel and the vertical pixel channel of the image after interpolation, after correcting the errors of the horizontal pixel interpolation and the vertical pixel interpolation of the G channel by Unet model, the horizontal pixel interpolation and the vertical pixel interpolation of the G channel after correcting the errors are fused, and the accurate green channel image of the single-channel gray level image is obtained.

The single-channel gray-scale image is demosaiced by using an interpolation algorithm and Unet models, for example, the method can be applied to digital image demosaicing technology developed for ISP chips in terminals, and the single-channel gray-scale image in a Bayer array mode is restored to be a complete red, blue and green three-channel color image.

Fig. 2 is a schematic diagram of a Unet network architecture employing the present disclosure. In fig. 2, since the Unet model only needs to blend the green channel horizontal direction pixel interpolation and the vertical direction pixel interpolation, the number of convolution boxes of the Unet model only needs to be 6. The image is input Unet to the model, then passed through an encoder (decoder), and then output through a decoder (decoder). The encoder (decoder) section includes three layers of convolution boxes (conv blocks), and then the image is passed by the encoder through the decoder section correspondingly also includes three layers of convolution boxes (conv blocks). There may be a short cut connection (short cut) between the encoder and decoder to increase Unet model learning performance on image features.

In addition, in order to enhance the invariance of the image scale, up-down sampling is added between every two convolution boxes of the Unet model, namely when the pixel interpolation in the horizontal direction and the pixel interpolation in the vertical direction of the green channel are fused through the Unet model, the amplified single-channel gray-scale image obtained by amplifying the single-channel gray-scale image and the abbreviated single-channel gray-scale image obtained by shrinking the single-channel gray-scale image are added between every two convolution boxes, so that the image multi-scale fusion can be achieved when the pixel interpolation in the horizontal direction and the pixel interpolation in the vertical direction of the green channel are fused.

In step S14, the green channel image is used as a reference template, and the interpolation algorithm is used to complement the red channel pixels and the blue channel pixels of the single-channel gray level image, so as to obtain a color image.

In the method, interpolation of pixels in the horizontal direction and interpolation of pixels in the vertical direction of a green channel of a single-channel gray image are fused through a pre-trained neural network model, after an accurate green channel image is obtained, red channel pixels and blue channel pixels of the single-channel gray image are complemented by a residual interpolation algorithm according to the accurate green channel image, an accurate red channel image and an accurate blue channel image are obtained, and then the accurate red channel image and the accurate blue channel image are output to obtain a color image.

For example, the completion of red channel pixels of a single channel gray scale image using a residual interpolation algorithm may be implemented as follows:

And performing guide up-sampling by taking the green channel image fused by the Unet model as a reference image to obtain an initialized red channel image, performing operation on the initialized red channel image and red channel pixels of the single-channel gray level image to obtain a minimum Laplacian residual error, and finally adding the up-sampled residual error result back to the red channel pixels of the single-channel gray level image to obtain the red channel image of the single-channel gray level image.

And similarly, taking the green channel image fused by the Unet model as a reference image, and complementing the pixels of the blue channel to obtain a blue channel image of the single-channel gray level image. And combining the green channel image of the obtained single-channel gray level image, the blue channel image of the single-channel gray level image and the red channel image of the obtained single-channel gray level image to obtain an output color image.

In the exemplary embodiment of the disclosure, before interpolation of pixels in the horizontal direction and interpolation of pixels in the vertical direction of a green channel of a single-channel gray image are spliced, the interpolation of the pixels in the horizontal direction and the interpolation of the pixels in the vertical direction of the green channel of the single-channel gray image are corrected and fused by utilizing the self-adaptive interpolation characteristic of a pre-trained neural network model, so that a green channel image with high accuracy can be obtained, and further, a color image with accurate color is output by utilizing a residual interpolation algorithm according to the green channel image with high accuracy. By the method and the device, after the image is demosaiced, the color image with better image details and fewer pseudo colors can be output.

In the present disclosure, the neural network model may be trained prior to demosaicing a single channel gray scale image to obtain a color image using an interpolation algorithm and a pre-trained neural network model.

The present disclosure describes a neural network for training demosaicing using a neural network model Unet as an example.

FIG. 3 is a flowchart of a training Unet model, according to an example embodiment, as shown in FIG. 3, a training Unet model method includes the following steps.

In step S21, an image training set is determined.

Because the training set plays a vital role in training the neural network model, in order to make the demosaicing effect of the Unet model after training better, the image training set can be acquired pertinently according to the camera module which is applied to the demosaicing requirement. For example, according to the parameters installed on the terminal a camera module, the image training set suitable for the terminal a camera module can be determined in a targeted manner. Furthermore, the training is more directional, so that the performance of training Unet models is better.

Wherein the image training set comprises color training images. The colored training images can be based on one or more of high-definition images acquired by a public high-definition image set, high-frequency texture images, images shot by a specified camera module and images shot by a single-lens reflex camera.

The high definition image obtained through the common high definition image set can be used for initializing Unet model parameters, and fixing Unet model parameters to a certain range. The picture with high-frequency texture from the design can be used to improve the texture recovery capability of Unet models to images. The image shooting module which needs to be applied to demosaicing is designated by the image shooting module, and the shot image can be used for improving the robustness of Unet models. The image shot by the single-lens reflex camera can be used for improving the image quality enhancement capability of the Unet model, such as improving the noise reduction capability of the image and improving the texture definition of the image.

Therefore, when the Unet model is trained through the image training set, as the image training set is specially designed for being applied to the appointed camera module, the Unet model is trained according to the targeted image training set, the Unet model after training has obvious pertinence and directivity, and the image can be processed more quickly when the Unet model after training is deployed to the digital image processing platform to which the appointed camera module is applied. And based on demosaicing of the image in the appointed camera module, the color of the obtained color image is more real, the noise intensity is weakened, and further the color image output through the Unet model has higher definition, and the color reproducibility of the image can still be better in extremely dark and extremely bright extreme illumination environments.

In step S22, a comparison graph of the single-channel gray scale training image and the Unet model output image is obtained according to the image training set.

In one embodiment, the present disclosure may preprocess color training images in an image training set to obtain preprocessed color training images. The preprocessing may include, for example, at least one of degamma and debalancing of the color training image.

In order to increase the training speed of the Unet model, the present disclosure may cut the preprocessed color training image into small images, for example, the preprocessed color training image may be cut into small images with pixel sizes of 30×30, to obtain a comparison image of the Unet model output image. And if the resolution of the preprocessed color training image exceeds the maximum value required by the training appointed camera module, for example, the image shot by the single-lens reflex camera or the high-definition image acquired based on the public high-definition image set can be cut after being downsampled into a proper size. For example, downsampling the image into an image with a pixel size of 3000 x 4000.

After a contrast image of Unet model output images is obtained, mosaic processing is carried out on the contrast image, and a single-channel gray training image corresponding to the contrast image is obtained, wherein the single-channel gray training image is input into the U-shaped neural network.

In one embodiment, in order to enable the trained Unet model to be demosaiced and simultaneously enhance an image and reduce noise, the present disclosure may further perform brightness and noise removal on the contrast map and perform edge enhancement processing to obtain a processed contrast map. And training Unet a model based on the processed contrast map and the single-channel gray scale training image.

In step S23, a U-shaped neural network model is trained based on the single-channel gray scale training image and the contrast map.

Inputting the single-channel gray scale training image and the contrast map into a Unet model, and predicting the color image of the single-channel gray scale training image through the Unet model to obtain a color prediction image corresponding to the single-channel gray scale training image. And calculating errors between the color prediction image and the contrast image according to the loss function, and adjusting Unet parameters of the model according to the calculated errors until the error calculated by the loss function is lower than a preset threshold value, so as to obtain the trained U-shaped neural network model.

In the exemplary embodiment of the disclosure, when the Unet model is trained by acquiring the image training set with the specific camera module, because the image training set is specially designed for being applied to the specific camera module, the Unet model is trained according to the specific image training set, so that the trained Unet model has obvious pertinence and directivity, and the image can be processed more quickly when the trained Unet model is deployed on the digital image processing platform with the specific camera module. And based on demosaicing of the image in the appointed camera module, the color of the obtained color image is more real, the noise intensity is weakened, and further the color image output through the Unet model has higher definition, and the color reproducibility of the image can still be better in extremely dark and extremely bright extreme illumination environments.

Based on the same inventive concept, the present disclosure also provides an image processing apparatus.

It may be understood that, in order to implement the above-mentioned functions, the application control device provided in the embodiments of the present disclosure includes a hardware structure and/or a software module that perform each function. The disclosed embodiments may be implemented in hardware or a combination of hardware and computer software, in combination with the various example elements and algorithm steps disclosed in the embodiments of the disclosure. Whether a function is implemented as hardware or computer software driven hardware depends upon the particular application and design constraints imposed on the solution. Those skilled in the art may implement the described functionality using different approaches for each particular application, but such implementation is not to be considered as beyond the scope of the embodiments of the present disclosure.

Fig. 4 is a block diagram 400 of an image processing apparatus according to an exemplary embodiment. Referring to fig. 4, the image processing apparatus includes an acquisition unit 401 and a processing unit 402.

The image processing device comprises an acquisition unit 401, a processing unit 402 and an interpolation algorithm, wherein the acquisition unit 401 is configured to acquire a single-channel gray image, the processing unit 402 is configured to interpolate green channel pixels of the single-channel gray image through the interpolation algorithm to obtain green channel horizontal direction pixel interpolation and green channel vertical direction pixel interpolation of the single-channel gray image, the pre-trained neural network model is utilized to adaptively interpolate and fuse the green channel horizontal direction pixel interpolation and the green channel vertical direction pixel interpolation to obtain the green channel image of the single-channel gray image, the green channel image is used as a reference template, and the interpolation algorithm is utilized to complement red channel pixels and blue channel pixels of the single-channel gray image to obtain the color image.

In an example, the image processing apparatus further comprises a determining unit 403 configured to determine an image training set, the image training set comprising color training images, the processing unit 402 further configured to pre-process the color training images in the image training set to obtain pre-processed color training images, determine the pre-processed color training images as a contrast map of the neural network model output image, mosaic the contrast map to obtain a single-channel gray scale training image corresponding to the contrast map, and the training unit 404 configured to train the neural network model based on the single-channel gray scale training image and the contrast map.

In one example, the image training set includes one or more of a high definition image, a high frequency texture image, an image captured by a designated camera module, and an image captured by a single lens reflex camera acquired based on a common high definition image set.

In an example, the processing unit 402 is further configured to perform brightness and noise removal on the contrast map and perform edge enhancement processing to obtain a processed contrast map.

In one example, the pre-trained neural network model is a U-shaped neural network model.

The specific manner in which the various modules perform the operations in the apparatus of the above embodiments have been described in detail in connection with the embodiments of the method, and will not be described in detail herein.

Fig. 5 is a block diagram illustrating an apparatus 500 for image processing according to an exemplary embodiment. For example, the apparatus 500 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, an exercise device, a personal digital assistant, or the like.

Referring to FIG. 5, the apparatus 500 may include one or more of a processing component 502, a memory 504, a power supply component 505, a multimedia component 508, an audio component 510, an input/output (I/O) interface 512, a sensor component 514, and a communication component 516.

The processing component 502 generally controls overall operation of the apparatus 500, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 502 may include one or more processors 520 to execute instructions to perform all or part of the steps of the methods described above. Further, the processing component 502 can include one or more modules that facilitate interactions between the processing component 502 and other components. For example, the processing component 502 can include a multimedia module to facilitate interaction between the multimedia component 508 and the processing component 502.

The memory 504 is configured to store various types of data to support operations at the apparatus 500. Examples of such data include instructions for any application or method operating on the apparatus 500, contact data, phonebook data, messages, pictures, videos, and the like. The memory 504 may be implemented by any type or combination of volatile or nonvolatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.

The power supply assembly 505 provides power to the various components of the apparatus 500. The power supply components 505 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the apparatus 500.

The multimedia component 508 includes a screen between the device 500 and the user that provides an output interface. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may sense not only the boundary of a touch or slide action, but also the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 508 includes a front-facing camera and/or a rear-facing camera. The front-facing camera and/or the rear-facing camera may receive external multimedia data when the apparatus 500 is in an operational mode, such as a photographing mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

The audio component 510 is configured to output and/or input audio signals. For example, the audio component 510 includes a Microphone (MIC) configured to receive external audio signals when the device 500 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may be further stored in the memory 504 or transmitted via the communication component 516. In some embodiments, the audio component 510 further comprises a speaker for outputting audio signals.

The I/O interface 512 provides an interface between the processing component 502 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, a home button, a volume button, an activate button, and a lock button.

The sensor assembly 514 includes one or more sensors for providing status assessment of various aspects of the apparatus 500. For example, the sensor assembly 514 may detect the on/off state of the device 500, the relative positioning of the components, such as the display and keypad of the device 500, the sensor assembly 514 may also detect a change in position of the device 500 or a component of the device 500, the presence or absence of user contact with the device 500, the orientation or acceleration/deceleration of the device 500, and a change in temperature of the device 500. The sensor assembly 514 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 514 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 514 may also include an acceleration sensor, a gyroscopic sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 516 is configured to facilitate communication between the apparatus 500 and other devices in a wired or wireless manner. The apparatus 500 may access a wireless network based on a communication standard, such as WiFi,2G or 3G, or a combination thereof. In one exemplary embodiment, the communication component 516 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 516 further includes a Near Field Communication (NFC) module to facilitate short range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra Wideband (UWB) technology, bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, the apparatus 500 may be implemented by one or more Application Specific Integrated Circuits (ASICs), digital Signal Processors (DSPs), digital Signal Processing Devices (DSPDs), programmable Logic Devices (PLDs), field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic elements for executing the methods described above.

In an exemplary embodiment, a non-transitory computer readable storage medium is also provided, such as memory 504, including instructions executable by processor 520 of apparatus 500 to perform the above-described method. For example, the non-transitory computer readable storage medium may be ROM, random Access Memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

It is further understood that the term "plurality" in this disclosure means two or more, and other adjectives are similar thereto. "and/or" describes an association relationship of an association object, and indicates that there may be three relationships, for example, a and/or B, and may indicate that there are three cases of a alone, a and B together, and B alone. The character "/" generally indicates that the context-dependent object is an "or" relationship. The singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

It is further understood that the terms "first," "second," and the like are used to describe various information, but such information should not be limited to these terms. These terms are only used to distinguish one type of information from another and do not denote a particular order or importance. Indeed, the expressions "first", "second", etc. may be used entirely interchangeably. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present disclosure.

It will be further understood that although operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any adaptations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It is to be understood that the present disclosure is not limited to the precise arrangements and instrumentalities shown in the drawings, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims (12)

1. An image processing method, characterized in that the method comprises: Get a single-channel grayscale image; Interpolating the green channel pixels of the single-channel grayscale image by an interpolation algorithm to obtain horizontal pixel interpolation of the green channel and vertical pixel interpolation of the green channel of the single-channel grayscale image; Using a pre-trained neural network model, adaptively interpolating and fusing the horizontal pixel interpolation of the green channel and the vertical pixel interpolation of the green channel to obtain a green channel image of the single-channel grayscale image; Taking the green channel image as a reference template, the interpolation algorithm is used to complement the red channel pixels and the blue channel pixels of the single-channel grayscale image to obtain the red channel image and the blue channel image of the single-channel grayscale image, and a color image is obtained based on the green channel image, the red channel image and the blue channel image of the single-channel grayscale image. 2. The image processing method according to claim 1, characterized in that the method further comprises: Determine an image training set, wherein the image training set includes color training images; Preprocessing the color training images in the image training set to obtain preprocessed color training images; Determine the preprocessed color training image as a comparison image of the output image of the neural network model; Performing mosaic processing on the comparison image to obtain a single-channel grayscale training image corresponding to the comparison image; The neural network model is trained based on the single-channel grayscale training image and the comparison image. 3. The image processing method according to claim 2, wherein the image training set comprises: One or more of high-definition images acquired based on a public high-definition image set, high-frequency texture images, images taken by a designated camera module, and images taken by a single-lens reflex camera. 4. The image processing method according to claim 2 is characterized in that the comparison image is a comparison image obtained after brightness and noise removal and edge enhancement processing. 5. The image processing method according to any one of claims 1 to 4, characterized in that the neural network model is a U-type neural network model. 6. An image processing device, characterized in that the device comprises: An acquisition unit is configured to acquire a single-channel grayscale image; a processing unit configured to interpolate the green channel pixels of the single-channel grayscale image through an interpolation algorithm to obtain the green channel horizontal pixel interpolation and the green channel vertical pixel interpolation of the single-channel grayscale image, and Using a pre-trained neural network model, adaptively interpolating and fusing the horizontal pixel interpolation of the green channel and the vertical pixel interpolation of the green channel to obtain a green channel image of the single-channel grayscale image, and Taking the green channel image as a reference template, the interpolation algorithm is used to complement the red channel pixels and the blue channel pixels of the single-channel grayscale image to obtain the red channel image and the blue channel image of the single-channel grayscale image, and a color image is obtained based on the green channel image, the red channel image and the blue channel image of the single-channel grayscale image. 7. The image processing device according to claim 6, characterized in that the device further comprises: A determination unit, configured to determine an image training set, wherein the image training set includes color training images; The processing unit is further configured to: Preprocessing the color training images in the image training set to obtain preprocessed color training images; Determine the preprocessed color training image as a comparison image of the output image of the neural network model; Performing mosaic processing on the comparison image to obtain a single-channel grayscale training image corresponding to the comparison image; A training unit is configured to train the neural network model based on the single-channel grayscale training image and the comparison image. 8. The image processing device according to claim 7, wherein the image training set comprises: One or more of high-definition images acquired based on a public high-definition image set, high-frequency texture images, images taken by a designated camera module, and images taken by a single-lens reflex camera. 9 . The image processing device according to claim 7 , wherein the comparison image is a comparison image obtained after brightness and noise are removed and edge enhancement is performed. 10. The image processing device according to any one of claims 6 to 9, characterized in that the pre-trained neural network model is a U-type neural network model. 11. An image processing device, comprising: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to: execute the image processing method according to any one of claims 1-5. 12. A non-transitory computer-readable storage medium, characterized in that the non-transitory computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the image processing method described in any one of claims 1 to 5 is executed.