CN106572289B - Image processing method of camera module and mobile terminal - Google Patents

Abstract

The embodiment of the invention provides a kind of image processing method of camera module and mobile terminals, the camera module are provided with imaging sensor, described image sensor has RGB output channel and IR output channel, which comprises acquisition incident ray；Described image sensor induction incident ray and conversion form electric image signal, include RGB image electric signal and IR electric image signal in described image electric signal；Described image sensor isolates RGB image electric signal and IR electric image signal from described image electric signal；The RGB output channel exports the RGB image electric signal, and, the IR output channel exports the IR electric image signal；Target image is obtained according to the RGB image electric signal and the IR electric image signal.The embodiment of the present invention is to solve the problems, such as that dual camera excessively occupies mobile terminal space.

Description

Image processing method of camera module and mobile terminal

technical field

The invention relates to the field of communication technology, in particular to an image processing method of a camera module and a mobile terminal.

Background technique

With the continuous development of electronic products, mobile terminals with a shooting function are becoming more and more popular, and users can use them to take pictures anytime and anywhere. In order to obtain better shooting effects, some mobile terminals currently on the market are equipped with dual rear cameras.

The dual rear camera means that two camera modules are installed on the back of the mobile terminal. The two camera modules can improve the shooting effect of the mobile terminal. However, since the two camera modules in the dual rear cameras are independent of each other, they do not affect each other when performing shooting work, that is to say, each of the two camera modules requires a complete set of components, such as lenses, image sensors (sensors) ), mirror mounts, motors, substrates, etc. It can be understood that the camera module with this structure will occupy more space on the mobile terminal, which may result in restrictions on some functions of the mobile terminal, or damage the appearance of the mobile terminal.

Contents of the invention

Embodiments of the present invention provide an image processing method of a camera module and a mobile terminal, so as to solve the problem that dual cameras occupy too much space of the mobile terminal.

In the first aspect, a method for image processing of a camera module is provided, the method is applied to a mobile terminal, an image sensor is provided on the camera module, and the image sensor has an RGB output path and an IR output path, so The methods described include:

collect incident light;

The image sensor senses incident light and converts it into an image electrical signal, and the image electrical signal includes an RGB image electrical signal and an IR image electrical signal;

The image sensor separates RGB image electrical signals and IR image electrical signals from the image electrical signals;

The RGB output path outputs the RGB image electrical signal, and the IR output path outputs the IR image electrical signal;

A target image is obtained according to the RGB image electrical signal and the IR image electrical signal.

In a second aspect, a mobile terminal is provided, the mobile terminal includes a camera module, an image sensor is provided on the camera module, the image sensor has an RGB output path and an IR output path, and the mobile terminal includes:

The incident light collection module is used to collect the incident light;

An image electrical signal conversion module, used for the image sensor to sense incident light and convert it into an image electrical signal, the image electrical signal including RGB image electrical signals and IR image electrical signals;

An image electrical signal separation module, used for the image sensor to separate RGB image electrical signals and IR image electrical signals from the image electrical signals;

An image electrical signal output module, configured for the RGB output channel to output the RGB image electrical signal, and the IR output channel to output the IR image electrical signal;

A target image generation module, configured to obtain a target image according to the RGB image electrical signal and the IR image electrical signal.

In this way, in the embodiment of the present invention, an image sensor is provided in the same camera module of the mobile terminal. The image sensor includes an RGB output channel and an IR output channel, and can output RGB image electrical signals and IR image electrical signals respectively. When the incident light is projected to the camera module, the incident light can be sensed by the image sensor and converted into an image electrical signal, and the RGB image electrical signal and the IR image electrical signal are separated from the image electrical signal, and the RGB image electrical signal is output through the corresponding output channel and the IR image electrical signal, and finally the target image can be obtained based on the RGB image data and the IR image data. By applying the embodiment of the present invention, the functions of two camera modules are realized in the same camera module, and image data can be independently output on the basis of hardware without processing by software. In addition, the embodiment of the present invention can realize the functions of the dual-camera module through one camera module. This simple structure not only effectively saves the space occupied by the existing dual-camera module on the mobile terminal, but also can simultaneously It has the functions of the dual camera module.

Description of drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the accompanying drawings that need to be used in the description of the embodiments of the present invention. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention , for those skilled in the art, other drawings can also be obtained according to these drawings without paying creative labor.

FIG. 1 is a flow chart of the steps of an embodiment of an image processing method for a camera module according to Embodiment 1 of the present invention;

2 is a flow chart of the steps of an embodiment of an image processing method for a camera module according to Embodiment 2 of the present invention;

FIG. 3 is a structural block diagram of a mobile terminal according to Embodiment 3 of the present invention;

FIG. 3a is one of the structural block diagrams of a mobile terminal according to Embodiment 3 of the present invention;

FIG. 3b is the second structural block diagram of a mobile terminal according to Embodiment 3 of the present invention;

FIG. 3c is a third structural block diagram of a mobile terminal according to Embodiment 3 of the present invention;

FIG. 4 is a block diagram of a mobile terminal according to Embodiment 4 of the present invention;

FIG. 5 is a schematic structural diagram of a mobile terminal according to Embodiment 5 of the present invention.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some of the embodiments of the present invention, but not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

Embodiment one

Please refer to FIG. 1 , which is a flowchart of steps of an image processing method for a camera module according to Embodiment 1 of the present invention.

An image processing method for a camera module provided in an embodiment of the present invention is applied to a mobile terminal. An image sensor is provided on the camera module, and the image sensor has an RGB output path and an IR output path. The method may specifically include Follow the steps below:

Step 101, collecting incident light.

In a specific implementation, the camera module includes a lens, and when light is reflected from the subject and incident on the camera module, the lens in the camera module collects the incident light of the subject for subsequent processing operations.

Step 102, the image sensor senses the incident light and converts it into an image electrical signal, and the image electrical signal includes an RGB image electrical signal and an IR image electrical signal.

In the camera module of the embodiment of the present invention, there may be only one image sensor, which can convert the sensed incident light into original image electrical signals including RGB image electrical signals and IR image electrical signals. The image sensor has at least two output channels, one is an RGB output channel and the other is an IR output channel, and these two channels can respectively output RGB image electrical signals and IR image electrical signals.

An image sensor, also called a photosensitive element, is a device that converts light signals into electronic signals, and is one of the important components of the camera module provided by the embodiments of the present invention. The image sensor provided by the embodiment of the present invention includes two types of CCD (Charge Coupled Device, charge coupled device) and CMOS (Complementary Metal-Oxide Semiconductor, metal oxide semiconductor element). Specifically, the above-mentioned CCD provides good image quality, anti-noise capability and flexibility in camera module design; however, it needs to add external circuits to increase the size and complexity of the system, and it needs to be more flexible in circuit design. To improve the performance of the CCD camera module as much as possible. The above-mentioned CCD is more suitable for applications that require very high camera module performance and less stringent cost control, such as astronomy, high-definition medical X-ray images, and other scientific applications that require long exposures and strict image noise requirements .

The above-mentioned CMOS is an image sensor produced by applying the contemporary large-scale semiconductor integrated circuit production process, which has the characteristics of high yield, high integration, low power consumption, and low price. The level of CMOS in image quality can be compared with CCD technology, and they are more suitable for applications requiring small space, small size, low power consumption and not particularly high requirements for image noise and quality. Such as most industrial inspection applications with auxiliary lighting, security and security applications, and most consumer commercial digital camera applications.

At present, the image sensor of the camera module on the mobile terminal mainly uses CMOS, and the CMOS will be introduced below.

CMOS is a typical solid-state imaging sensor, usually composed of pixel array, row driver, column driver, timing control logic, A/D converter, data bus output interface, control interface, etc. These parts are usually integrated on the same silicon wafer.

The basic working principle of CMOS is as follows: First, the incident light from the outside illuminates the pixel array, a photoelectric effect occurs, and corresponding charges are generated in the pixel units of the pixel array to be converted into electrical signals. The CMOS row selection logic unit gates the corresponding row of pixel units as required. The image electrical signals in the row pixel units are transmitted to the corresponding analog signal processing units and A/D converters through the signal buses of the respective columns, converted into digital image electrical signals, and then output through corresponding output channels. The row selection logic unit can scan the pixel array progressively or interlacedly. The row selection logic unit cooperates with the column selection logic unit to realize the window extraction function of the image.

Of course, other image sensors other than CMOS and CCD image sensors may also be used in the embodiment of the present invention, as long as the image electrical signal can be collected, the embodiment of the present invention does not limit this.

Step 103, the image sensor separates RGB image electrical signals and IR image electrical signals from the image electrical signals.

In a specific implementation, some pixels on the image sensor may be skipped and ignored. The embodiment of the present invention adopts this method, so that an image sensor can be used to separate the required RGB image electrical signal and IR image electrical signal from the image electrical signal, that is, the separation of the image electrical signal can be realized on the hardware. It does not need to be realized by means of software control.

Step 104, the RGB output path outputs the RGB image electrical signal, and the IR output path outputs the IR image electrical signal.

In a specific application, the image sensor may include two output paths, so through the RGB output path and the IR output path of an image sensor, the RGB image electrical signal and the IR image electrical signal separated from the image electrical signal can be respectively output. The data of the RGB image electrical signal and the IR image electrical signal are respectively stored in designated storage areas, and can be extracted and used when necessary.

Step 105, obtaining a target image according to the electrical RGB image signal and the electrical IR image signal.

In the specific application of the present invention, for the obtained RGB image electrical signals and IR image electrical signals, the corresponding image electrical signals can be selected to be processed individually or comprehensively according to actual needs, for example, converted into digital signals by a digital-to-analog converter chip , to form the desired target image. Based on the obtained target image, functions such as conventional image capture, perception of space and depth of field, iris, and creation of 3D images can be realized.

In this way, in the embodiment of the present invention, an image sensor is provided in the same camera module of the mobile terminal. The image sensor includes an RGB output channel and an IR output channel, and can output RGB image electrical signals and IR image electrical signals respectively. When the incident light is projected to the camera module, the incident light can be sensed by the image sensor and converted into an image electrical signal, and the RGB image electrical signal and the IR image electrical signal are separated from the image electrical signal, and the RGB image electrical signal is output through the corresponding output channel and the IR image electrical signal, and finally the target image can be obtained based on the RGB image data and the IR image data. By applying the embodiment of the present invention, the functions of two camera modules are realized in the same camera module, and image data can be independently output on the basis of hardware without processing by software. In addition, the embodiment of the present invention can realize the functions of the dual-camera module through one camera module. This simple structure not only effectively saves the space occupied by the existing dual-camera module on the mobile terminal, but also can simultaneously It has the functions of the dual camera module.

Embodiment two

Please refer to FIG. 2 , which is a flowchart of steps of an image processing method for a camera module according to Embodiment 2 of the present invention.

An image processing method for a camera module provided in an embodiment of the present invention is applied to a mobile terminal. An image sensor is provided on the camera module, and the image sensor has an RGB output path and an IR output path. The method may specifically include Follow the steps below:

Step 201, collecting incident light.

Step 202, the image sensor senses the incident light and converts it into an electrical image signal, and the electrical image signal includes an electrical RGB image signal and an electrical IR image signal.

Since the steps 201-202 of the second embodiment are similar to those of the first method embodiment, for the details in the description of the steps 201-202 in this embodiment, you can refer to the relevant descriptions in the first method embodiment above, and will not repeat them here. up.

Step 203, the image sensor separates RGB image electrical signals and IR image electrical signals from the image electrical signals.

In a specific implementation, the image sensor can realize screening for some specific pixels, that is, the image sensor can only filter the RGB image electrical signal from the image electrical signal, and the image sensor can only filter out the IR image electrical signal from the image electrical signal , so as to realize the separation of the RGB image electrical signal and the IR image electrical signal in the image electrical signal, for example:

In an embodiment of the present invention, the IR image electrical signal can be screened out from the image data, specifically refer to the following sub-steps in step 203:

The image sensor removes RGB image electrical signals row by row from the image electrical signals.

The image sensor removes RGB image electrical signals from the image electrical signals column by column.

The image electrical signal from which the RGB image electrical signals have been removed is used as an IR image electrical signal.

In the image sensor, desampling can be performed on the entire row and column, that is to say, the IR image electrical signal can be obtained separately by combining row sampling and column sampling. Specifically, the rows without IR image electrical signals can be removed from the image electrical signals first, and then the columns without IR image electrical signals can be removed, so that there is no RGB image electrical signals in the image electrical signals, and finally the IR image electrical signals can be obtained.

In an embodiment of the present invention, the RGB image electrical signal can be filtered out from the image data, specifically referring to the sub-steps shown in step 203 as follows:

extracting a G component around the IR image electrical signal from the image electrical signal;

calculating an average G component based on the surrounding G components;

Using the average G component to replace the IR image electrical signal of the line;

The substituted electrical image signal is used as an electrical RGB image signal.

As shown in the pixel arrangement of RGB-IR image data in the above figure, RGB data cannot be obtained by removing entire rows and columns like IR data, but an image data device can be added inside the sensor, and then the image algorithm can be used to replace the IR pixels , if the G of the surrounding pixels can be averaged, the obtained G can be used to complement the IR pixel, and finally the RGB image electrical signal can be obtained.

Step 204, the RGB output path outputs the RGB image electrical signal, and the IR output path outputs the IR image electrical signal.

In the embodiment of the present invention, there may be only one image sensor in the camera module, and the image sensor has multiple output channels, which can respectively output RGB image electrical signals and IR image electrical signals correspondingly.

When the image sensor generates an electrical image signal, and samples an electrical image signal of RGB and an electrical image signal of IR from the electrical image signal, the electrical image signal can be output through corresponding output channels respectively. For example, the RGB image electrical signal is output from the RGB output channel, and the IR image electrical signal is output through the IR output channel.

Since the image sensor in the same camera module has at least two output channels to output different image electrical signals, that is, the camera module hardware can realize two outputs, that is, it can output RGB image electrical signals and IR image electrical signals at the same time, so that the same The camera module can realize the functions of two completely independent cameras without software processing.

Step 205, obtain a color target RGB image based on the RGB image electrical signal, or obtain a black and white target IR image based on the IR image electrical signal, or obtain a black and white target IR image based on the RGB image electrical signal and the IR image electrical signal Target blended image.

The camera module of the embodiment of the present invention can collect two-way image electrical signals, that is, RGB image electrical signals and IR image electrical signals, which is equivalent to having two cameras, with clearer shooting, shallow depth of field shooting and better low-light performance And so on.

Specifically, the first function of the camera module in the embodiment of the present invention is to support shallow depth of field photography, realize background blurring, and highlight the photographed object; the second function is to have better performance in low-light environments. Through the camera module of the embodiment of the present invention, it is possible to analyze which pixel has a higher degree of restoration of the scene, which can help the mobile terminal to effectively suppress noise, so that the image picture has better image quality restoration.

For example, the embodiment of the present invention uses the RGB image electrical signal to obtain a conventional color image. Since the IR image electrical signal is removed, the image based on the RGB image electrical signal will not be red and distorted; it is also possible to use the IR image electrical signal to obtain a black and white image. Based on the characteristics of infrared rays, the obtained black-and-white image can perceive space and depth of field; further, a mixed image can be obtained by using RGB image electrical signals and IR image electrical signals at the same time, and the image combined with the characteristics of the two can be a 3D image that can reflect spatial information image.

Certainly, the RGB image electrical signal and the IR image electrical signal separated by the camera module in the embodiment of the present invention may be applied according to actual needs, and the embodiment of the present invention does not limit this.

In the embodiment of the present invention, through an image sensor in the camera module of the mobile terminal, the separation and output of the RGB image electrical signal and the IR image electrical signal in the image electrical signal can be realized on the hardware, without the need for software processing , so that the function of dual cameras can be realized in one camera module.

Embodiment three

Please refer to FIG. 3 , which is a structural block diagram of a mobile terminal 300 according to Embodiment 3 of the present invention.

In a mobile terminal provided by an embodiment of the present invention, the mobile terminal may include a camera module, an image sensor is provided on the camera module, the image sensor has an RGB output path and an IR output path, and the mobile terminal may It includes an incident light collection module 301 , an image electrical signal conversion module 302 , an image electrical signal separation module 303 , an image electrical signal output module 304 and a target image generation module 305 .

The incident light collection module 301 is configured to collect incident light.

The electrical image signal conversion module 302 is used for the image sensor to sense incident light and convert it into electrical image signals, the electrical image signals include electrical RGB image signals and electrical IR image signals.

The image electrical signal separation module 303 is used for the image sensor to separate RGB image electrical signals and IR image electrical signals from the image electrical signals.

The image electrical signal output module 304 is used for the RGB output path to output the RGB image electrical signal, and the IR output path to output the IR image electrical signal.

A target image generating module 305, configured to obtain a target image based on the RGB image electrical signal and the IR image electrical signal.

3a, in a mobile terminal 300 according to an embodiment of the present invention, the image electrical signal separation module 303 includes a first removal submodule 3031, a second removal submodule 3032 and an IR image electrical signal acquisition submodule 3033.

The first removing sub-module 3031 is used for the image sensor to remove RGB image electrical signals from the image electrical signals row by row;

The second removing sub-module 3032 is used for the image sensor to remove RGB image electrical signals from the image electrical signals column by column;

The IR image electrical signal obtaining sub-module 3033 is configured to use the image electrical signal from which the RGB image electrical signals have been removed as an IR image electrical signal.

Referring to Figure 3b, in a mobile terminal 300 according to an embodiment of the present invention, the image electrical signal separation module 303 includes a G component extraction submodule 3034, an average G component calculation submodule 3035, an IR image electrical signal substitution submodule 3036 and an RGB Image electrical signal acquisition sub-module 3037.

The G component extraction sub-module 3034 is used to extract the G component around the IR image electrical signal from the image electrical signal;

The average G component calculation sub-module 3035 is used to calculate the average G component based on the surrounding G components;

The IR image electrical signal sub-module 3036 is used to replace the IR image electrical signal of the line with the average G component;

The RGB image electrical signal obtaining sub-module 3037 is configured to use the substituted image electrical signal as an RGB image electrical signal.

As shown in the pixel arrangement of RGB-IR image data in the above figure, RGB data cannot be obtained by removing entire rows and columns like IR data, but an image data processor can be added inside the sensor, and then the IR pixels can be processed using image algorithms For replacement, for example, the G of surrounding pixels can be averaged, and the obtained G can be used to complement the IR pixel. Finally, RGB image electrical signals are obtained.

Referring to FIG. 3 c , in a mobile terminal 300 according to an embodiment of the present invention, the target image generation module 305 includes a target RGB image acquisition submodule 3051 , a target IR image acquisition submodule 3052 and a target mixed image acquisition submodule 3053 .

The target RGB image obtaining sub-module 3051 is used to obtain a colored target RGB image based on the electrical signal of the RGB image;

The target IR image obtaining sub-module 3052 is used to obtain a black and white target IR image based on the electrical signal of the IR image;

The target mixed image obtaining sub-module 3053 is configured to obtain a target mixed image based on the electrical RGB image signal and the electrical IR image signal.

In a mobile terminal 300 according to an embodiment of the present invention, the number of image sensors provided in the camera module is one.

As for the embodiment of the mobile terminal 300 , since it is basically similar to the method embodiment, the description is relatively simple, and for related parts, refer to the part of the description of the method embodiment.

In this way, in the embodiment of the present invention, an image sensor is provided in the same camera module of the mobile terminal. The image sensor includes an RGB output channel and an IR output channel, and can output RGB image electrical signals and IR image electrical signals respectively. When the incident light is projected to the camera module, the incident light can be converted into an image electrical signal based on the image sensor, and the RGB image electrical signal and the IR image electrical signal are separated from the image electrical signal, and the RGB image electrical signal is output through the corresponding output channel and the IR image electrical signal, and finally the target image can be obtained based on the RGB image data and the IR image data. Applying the embodiment of the present invention, there are equivalent to two camera modules in the same camera module, which can realize independent output of image data on the basis of hardware, without processing by software. In addition, the embodiment of the present invention can realize the functions of the dual-camera module through one camera module. This simple structure not only effectively saves the space occupied by the previous dual-camera module on the mobile terminal, but also retains the The functions of the dual camera module.

Embodiment four

FIG. 4 is a block diagram of a mobile terminal 700 according to another embodiment of the present invention.

The mobile terminal 700 shown in FIG. 4 includes: at least one processor 701 , a memory 702 , at least one network interface 704 and other user interfaces 703 . Various components in the mobile terminal 700 are coupled together through a bus system 705 . It can be understood that the bus system 705 is used to realize connection and communication between these components. In addition to the data bus, the bus system 705 also includes a power bus, a control bus and a status signal bus. However, for clarity of illustration, the various buses are labeled as bus system 705 in FIG. 4 .

Wherein, the user interface 703 may include a display, a keyboard or a pointing device (for example, a mouse, a trackball (trackball), a touch panel or a touch screen, and the like.

It can be understood that the memory 702 in the embodiment of the present invention may be a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memories. Among them, the non-volatile memory can be read-only memory (Read-OnlyMemory, ROM), programmable read-only memory (ProgrammableROM, PROM), erasable programmable read-only memory (ErasablePROM, EPROM), electrically erasable Programming read-only memory (Electrically EPROM, EEPROM) or flash memory. The volatile memory may be random access memory (Random Access Memory, RAM), which acts as an external cache. By way of illustration and not limitation, many forms of RAM are available such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (Synchronous DRAM, SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DoubleDataRateSDRAM, DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (Enhanced SDRAM, ESDRAM), Synchronous Link Dynamic Random Access Memory (SynchlinkDRAM, SLDRAM) and Direct Memory Bus Random Access Memory (Direct Rambus RAM, DRRAM). The memory 702 of the systems and methods described in embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

In some implementations, the memory 702 stores the following elements, executable modules or data structures, or their subsets, or their extended sets: an operating system 7021 and an application program 7022 .

Among them, the operating system 7021 includes various system programs, such as framework layer, core library layer, driver layer, etc., for realizing various basic services and processing hardware-based tasks. The application program 7022 includes various application programs, such as a media player (MediaPlayer), a browser (Browser), etc., and is used to implement various application services. The program for realizing the method of the embodiment of the present invention may be included in the application program 7022 .

In the embodiment of the present invention, by calling the program or instruction stored in the memory 702, specifically, the program or instruction stored in the application program 7022, the processor 701 is used to collect the incident light; the image sensor senses the incident light and converts Form electrical image signals, the electrical image signals include electrical RGB image signals and electrical IR image signals; the image sensor separates the electrical image signals RGB and IR image electrical signals from the electrical image signals; the RGB output path Outputting the electrical RGB image signal, and the IR output channel outputting the electrical IR image signal; obtaining a target image according to the electrical RGB image signal and the electrical IR image signal.

The methods disclosed in the foregoing embodiments of the present invention may be applied to the processor 701 or implemented by the processor 701 . The processor 701 may be an integrated circuit chip with signal processing capability. In the implementation process, each step of the above method may be completed by an integrated logic circuit of hardware in the processor 701 or instructions in the form of software. The above-mentioned processor 701 may be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), an off-the-shelf programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates Or transistor logic devices, discrete hardware components. Various methods, steps and logic block diagrams disclosed in the embodiments of the present invention may be implemented or executed. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like. The steps of the methods disclosed in the embodiments of the present invention may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the field such as random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, register. The storage medium is located in the memory 702, and the processor 701 reads the information in the memory 702, and completes the steps of the above method in combination with its hardware.

It can be understood that the embodiments described in the embodiments of the present invention may be implemented by hardware, software, firmware, middleware, microcode or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (Application Specific Integrated Circuits, ASIC), digital signal processor (Digital Signal Processing, DSP), digital signal processing device (DSPDevice, DSPD), programmable logic device (ProgrammableLogicDevice, PLD ), Field-Programmable Gate Array (Field-Programmable GateArray, FPGA), general-purpose processor, controller, microcontroller, microprocessor, other electronic units for performing the functions described in this application, or a combination thereof.

For software implementation, the techniques described in the embodiments of the present invention may be implemented through modules (such as procedures, functions, etc.) that execute the functions described in the embodiments of the present invention. Software codes can be stored in memory and executed by a processor. Memory can be implemented within the processor or external to the processor.

Optionally, the processor 701 is further configured to: the image sensor removes RGB image electrical signals from the image electrical signals row by row; the image sensor removes RGB image electrical signals from the image electrical signals column by column ; Using the electrical image signal from which the RGB image electrical signal has been removed, as an IR electrical image signal.

Optionally, the processor 701 is further configured to: extract a G component around the IR image electrical signal from the image electrical signal; calculate an average G component based on the surrounding G component; use the average G component to replace The IR image electrical signal of the line; the replaced image electrical signal is used as an RGB image electrical signal.

Optionally, the processor 701 is further configured to: obtain a color target RGB image based on the RGB image electrical signal; obtain a black and white target IR image based on the IR image electrical signal; and, based on the RGB image electrical signal and the The IR image electric signal is used to obtain the target mixed image.

The mobile terminal 700 can implement various processes implemented by the mobile terminal in the foregoing embodiments, and to avoid repetition, details are not repeated here.

In this way, in the embodiment of the present invention, an image sensor is provided in the same camera module of the mobile terminal. The image sensor includes an RGB output channel and an IR output channel, and can output RGB image electrical signals and IR image electrical signals respectively. When the incident light is projected to the camera module, the incident light can be sensed by the image sensor and converted into an image electrical signal, and the RGB image electrical signal and the IR image electrical signal are separated from the image electrical signal, and the RGB image electrical signal is output through the corresponding output channel and the IR image electrical signal, and finally the target image can be obtained based on the RGB image data and the IR image data. By applying the embodiment of the present invention, the functions of two camera modules are realized in the same camera module, and image data can be independently output on the basis of hardware without processing by software. In addition, the embodiment of the present invention can realize the functions of the dual-camera module through one camera module. This simple structure not only effectively saves the space occupied by the existing dual-camera module on the mobile terminal, but also can simultaneously It has the functions of the dual camera module.

Embodiment five

Fig. 5 is a schematic structural diagram of a mobile terminal according to another embodiment of the present invention. Specifically, the mobile terminal 800 in FIG. 5 may be a mobile phone, a tablet computer, a personal digital assistant (Personal Digital Assistant, PDA), or a vehicle-mounted computer.

Mobile terminal 800 in FIG.

Wherein, the input unit 830 can be used for receiving number or character information input by the user, and generating signal input related to the user setting and function control of the mobile terminal 800 . Specifically, in the embodiment of the present invention, the input unit 830 may include a touch panel 831 . The touch panel 831, also referred to as a touch screen, can collect user's touch operations on or near it (such as the user's operation on the touch panel 831 using any suitable object or accessory such as a finger or a stylus), and based on preset The specified program drives the corresponding connected device. Optionally, the touch panel 831 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, and detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and sends it to the to the processor 860, and can receive and execute commands sent by the processor 860. In addition, the touch panel 831 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 831, the input unit 830 may also include other input devices 832, which may include but not limited to physical keyboards, function keys (such as volume control buttons, switch buttons, etc.), trackballs, mice, joysticks, etc. one or more of.

Wherein, the display unit 840 can be used to display information input by the user or information provided to the user and various menu interfaces of the mobile terminal 800 . The display unit 840 may include a display panel 841. Optionally, the display panel 841 may be configured in the form of an LCD or an organic light-emitting diode (Organic Light-Emitting Diode, OLED).

It should be noted that the touch panel 831 can cover the display panel 841 to form a touch display screen. When the touch display screen detects a touch operation on or near it, it is sent to the processor 860 to determine the type of the touch event, and then the processor The 860 provides corresponding visual output on the touch display screen according to the type of the touch event.

The touch display screen includes an application program interface display area and a common control display area. The arrangement of the display area of the application program interface and the display area of the commonly used controls is not limited, and may be an arrangement in which the two display areas can be distinguished, such as vertical arrangement, left-right arrangement, and the like. The application program interface display area can be used to display the interface of the application program. Each interface may include at least one interface element such as an icon of an application program and/or a widget desktop control. The application program interface display area can also be an empty interface without any content. The commonly used control display area is used to display controls with a high usage rate, for example, application icons such as setting buttons, interface numbers, scroll bars, and phonebook icons.

Wherein the processor 860 is the control center of the mobile terminal 800, utilizes various interfaces and lines to connect the various parts of the whole mobile phone, by running or executing the software programs and/or modules stored in the first memory 821, and calling the software programs and/or modules stored in the second memory 821. The data in the memory 822 executes various functions of the mobile terminal 800 and processes data, so as to monitor the mobile terminal 800 as a whole. Optionally, the processor 860 may include one or more processing units.

In the embodiment of the present invention, by calling the software programs and/or modules stored in the first memory 821 and/or the data in the second memory 822, the processor 860 is used to collect incident light; the image sensor senses the incident light The light is converted into an image electrical signal, and the image electrical signal includes an RGB image electrical signal and an IR image electrical signal; the image sensor separates the RGB image electrical signal and the IR image electrical signal from the image electrical signal; the The RGB output path outputs the RGB image electrical signal, and the IR output path outputs the IR image electrical signal; a target image is obtained according to the RGB image electrical signal and the IR image electrical signal.

The processor 860 is further configured to: the image sensor removes RGB image electrical signals from the image electrical signals row by row; the image sensor removes RGB image electrical signals from the image electrical signals column by column; The image electrical signal from which the RGB image electrical signals have been removed is used as an IR image electrical signal.

The processor 860 is further configured to: extract the G component around the IR image electrical signal from the image electrical signal; calculate the average G component based on the surrounding G component; use the average G component to replace the line The IR image electrical signal; the replaced image electrical signal as an RGB image electrical signal.

The processor 860 is also configured to: obtain a color target RGB image based on the RGB image electrical signal; obtain a black and white target IR image based on the IR image electrical signal; and, based on the RGB image electrical signal and the IR The image electric signal obtains the target mixed image.

As for the embodiment of the mobile terminal 800, since it is basically similar to the method embodiment, the description is relatively simple, and for relevant parts, refer to the part of the description of the method embodiment.

In this way, in the embodiment of the present invention, an image sensor is provided in the same camera module of the mobile terminal. The image sensor includes an RGB output channel and an IR output channel, and can output RGB image electrical signals and IR image electrical signals respectively. When the incident light is projected to the camera module, the incident light can be sensed by the image sensor and converted into an image electrical signal, and the RGB image electrical signal and the IR image electrical signal are separated from the image electrical signal, and the RGB image electrical signal is output through the corresponding output channel and the IR image electrical signal, and finally the target image can be obtained based on the RGB image data and the IR image data. By applying the embodiment of the present invention, the functions of two camera modules are realized in the same camera module, and image data can be independently output on the basis of hardware without processing by software. In addition, the embodiment of the present invention can realize the functions of the dual-camera module through one camera module. This simple structure not only effectively saves the space occupied by the existing dual-camera module on the mobile terminal, but also can simultaneously It has the functions of the dual camera module.

Those of ordinary skill in the art can appreciate that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed in the embodiments of the present invention can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed by hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be regarded as exceeding the scope of the present invention.

Those skilled in the art can clearly understand that for the convenience and brevity of the description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

In the embodiments provided in this application, it should be understood that the disclosed devices and methods may be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or May be integrated into another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.

If the functions described above are realized in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the essence of the technical solution of the present invention or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media capable of storing program codes such as U disk, mobile hard disk, ROM, RAM, magnetic disk or optical disk.

The above is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Anyone skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present invention. Should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims (7)

1. a kind of image processing method of camera module is applied to mobile terminal, which is characterized in that in the camera module It is provided with imaging sensor, described image sensor has RGB output channel and IR output channel, which comprises

Acquire incident ray；

Described image sensor induction incident ray and conversion form electric image signal, include that RGB schemes in described image electric signal As electric signal and IR electric image signal；

Described image sensor isolates RGB image electric signal and IR electric image signal from described image electric signal；

The RGB output channel exports the RGB image electric signal, and, the IR output channel exports the IR image electricity Signal；

Target image is obtained according to the RGB image electric signal and the IR electric image signal；

Wherein, described image sensor isolates RGB image electric signal and IR electric image signal from described image electric signal Step includes:

From the G component extracted in described image electric signal around the IR electric image signal；

Average G component is calculated based on the G component around described；

The IR electric image signal is substituted using the average G component；

Using the electric image signal after the substitution as RGB image electric signal.

2. the method according to claim 1, wherein described according to RGB image electric signal and IR image electricity Signal obtains the step of target image and includes:

Colored target RGB image is obtained based on the RGB image electric signal；

The target IR image of black and white is obtained based on the IR electric image signal；

And target mixed image is obtained based on the RGB image electric signal and the IR electric image signal.

3. the method according to claim 1, wherein of the imaging sensor in camera module setting Number is one.

4. a kind of mobile terminal, which is characterized in that the mobile terminal includes camera module, is arranged in the camera module There is imaging sensor, described image sensor has RGB output channel and IR output channel, and the mobile terminal includes:

Incident ray acquisition module, for acquiring incident ray；

Electric image signal conversion module incudes incident ray for described image sensor and conversion forms electric image signal, institute State includes RGB image electric signal and IR electric image signal in electric image signal；

Electric image signal separation module isolates RGB image telecommunications for described image sensor from described image electric signal Number and IR electric image signal；

Electric image signal output module exports the RGB image electric signal for the RGB output channel, and, the IR is defeated Access exports the IR electric image signal out；

Target image generation module, for obtaining target image according to the RGB image electric signal and the IR electric image signal；

Wherein, described image electric signal separation module includes:

G component extraction submodule, for from the G component extracted in described image electric signal around the IR electric image signal；

Average G component computational submodule calculates average G component for the G component based on the surrounding；

IR electric image signal substitutes submodule, for substituting the IR electric image signal using the average G component；

RGB image electric signal obtains submodule, for using the electric image signal after the substitution as RGB image electric signal.

5. mobile terminal according to claim 4, which is characterized in that described image electric signal separation module includes:

First removes submodule, removes RGB image electric signal in described image electric signal line by line for described image sensor；

Second removes submodule, removes RGB image electric signal in described image electric signal by column for described image sensor；

IR electric image signal obtains submodule, for being schemed as IR by the electric image signal for having removed RGB image electric signal As electric signal.

6. mobile terminal according to claim 4, which is characterized in that the target image generation module includes:

Target RGB image obtains submodule, for obtaining colored target RGB image based on the RGB image electric signal；

Target IR image obtains submodule, for obtaining the target IR image of black and white based on the IR electric image signal；

Target mixed image obtains submodule, for obtaining mesh based on the RGB image electric signal and the IR electric image signal Mark mixed image.

7. mobile terminal according to claim 4, which is characterized in that in the imaging sensor of camera module setting Number be one.