CN115776606A - Sensor, camera module, electronic device and image processing method - Google Patents

Abstract

The application discloses a sensor, a camera module, electronic equipment and an image processing method, wherein the sensor includes: a filter unit, the filter unit includes RGB filters and a variety of narrow-band filters, and a variety of narrow-band filters The bandwidth of the chip is continuous in the preset spectral range, and the preset spectral range includes the visible spectrum; the pixel unit, the pixel unit includes a plurality of photosensitive pixels, and any photosensitive pixel of the pixel unit is set on the RGB filter or the narrow band of the filter unit One side of the filter; among them, the RGB filter is used to obtain RGB image information, and a variety of narrow-band filters are used to obtain multi-spectral information.

Description

Sensor, camera module, electronic device and image processing method

technical field

The present application belongs to the field of imaging technology, and specifically relates to a sensor, a camera module, electronic equipment and an image processing method.

Background technique

With the continuous development of imaging technology, the gap between the shooting function of electronic equipment and that of professional shooting equipment is gradually narrowing. In order to realize shooting and imaging, an image sensor is installed in the electronic device, and the image sensor can realize color screening through a color filter, and thereby capture and restore the ambient light to obtain an image of the environment where it is located. However, the image sensor in the related art can only record the brightness of the image at the moment of shooting, and ignores the influence of the light source in the environment scene on the image, so that the finally obtained image is quite different from the image perceived by human eyes.

Contents of the invention

The present application aims to provide a sensor, camera module, electronic equipment and image processing method, aiming to provide an image data acquisition scheme capable of obtaining real spectral information in the environment.

In the first aspect, the embodiment of the present application proposes a sensor, including:

A filter unit, the filter unit includes an RGB filter and a variety of narrow-band filters, the bandwidth of the various narrow-band filters is continuous in a preset spectral range, and the preset spectral range includes visible spectrum;

A pixel unit, the pixel unit includes a plurality of light-sensitive pixels, any one of the light-sensitive pixels of the pixel unit is arranged on one side of the RGB filter or the narrow-band filter;

Wherein, the RGB filter is used to obtain RGB image information, and the various narrow-band filters are used to obtain multi-spectral information.

In the second aspect, the embodiment of the present application provides a camera module, the camera module includes the sensor in the first aspect above.

In a third aspect, the embodiment of the present application provides an electronic device, where the electronic device includes the sensor in the first aspect above.

In the fourth aspect, the embodiment of the present application also proposes an image processing method, the image processing method is applied to the electronic device of the third aspect, and the method includes:

Obtaining the first image and multispectral information corresponding to the first image through the sensor when the shooting instruction is received;

Perform color correction on the first image according to the multispectral information.

In the embodiment of the present application, by setting the filter unit and the pixel unit in the sensor, wherein the filter unit includes RGB filters and various narrow-band filters, the photosensitive pixels in the pixel unit and the RGB filters Or the corresponding setting of the narrow-band filter, and because the filter unit is provided with a variety of narrow-band filters, and the bandwidth of the various narrow-band filters is continuous in the preset spectral range, and the preset spectral range includes the visible spectrum, The RGB filter can obtain RGB image information, and a variety of narrow-band filters can obtain multi-spectral information, thereby providing an image data acquisition solution for obtaining real spectral information in the environment, which can collect the light source in the environmental scene Information, so that the subsequent image processing obtained by shooting can be based on real spectral information, which indirectly improves the realism of the image and improves the difference between the image and the perception of the human eye.

Additional aspects and advantages of the application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the application.

Description of drawings

The above and/or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, wherein:

Fig. 1 is a schematic diagram of a sensor according to an embodiment of the present application;

Fig. 2 is a plan view of the filter in the sensor according to the embodiment of the application;

3 is a schematic diagram of frequency bands of various narrow-band filters in a sensor according to an embodiment of the present application;

FIG. 4 is a schematic flow chart of an image processing method according to an embodiment of the present application;

5 is a schematic diagram of spectral information and a first image obtained by an image processing method according to an embodiment of the present application;

FIG. 6 is a schematic diagram of a hardware structure of an electronic device provided according to an embodiment of the present application;

FIG. 7 is a schematic diagram of a hardware structure of another electronic device provided according to an embodiment of the present application.

Reference signs:

Filter unit 10; RGB filter 11; narrow band filter 12; basic unit block 13;

Pixel unit 20;

Red narrow-band filter R; green narrow-band filter G; blue narrow-band filter B.

Detailed ways

Embodiments of the present application will be described in detail below, and examples of the embodiments are shown in the drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary, and are only for explaining the present application, and should not be construed as limiting the present application. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of this application.

The features of the terms "first" and "second" in the description and claims of the present application may explicitly or implicitly include one or more of these features. In the description of the present application, unless otherwise specified, "plurality" means two or more. In addition, "and/or" in the specification and claims means at least one of the connected objects, and the character "/" generally means that the related objects are an "or" relationship.

In the description of this application, it is to be understood that the terms "center", "depth", "upper", "lower", "front", "rear", "left", "right", "vertical", The orientation or positional relationship indicated by "horizontal", "top", "bottom", "inner", "outer", etc. are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the application and simplifying the description, rather than Nothing to indicate or imply that a referenced device or element must have a particular orientation, be constructed, and operate in a particular orientation should therefore not be construed as limiting the application.

Multi-spectral technology is different from the traditional red, green and blue three-channel data acquisition method. It is a technology that divides the sampling spectrum according to a finer narrow-band bandpass, and restores and processes the collected spectral data.

At present, the application of multispectral technology is relatively limited, and the mainstream application directions can be roughly divided into two categories: one is for substance detection, such as sunscreen detection, sugar content of fruits, chemical composition of gases, etc. Limited, narrow application scope; the other is the application of expanding the dimension of traditional RGB images through spectral data, such as military monitoring, weather remote sensing, etc., which are mainly used in non-civilian fields.

These two types of applications can be divided into detection and imaging from the technical level. The detection needs to preprocess the detection samples first, form a database through sampling and classification of a large number of samples, and then pass professional clustering modeling and calibration. Landed in the application. The imaging applications are mainly concentrated in the long-distance imaging direction.

To sum up, these two types of applications are far away from our daily life, and their application value is limited.

In the imaging technology of electronic equipment, the relevant mainstream image sensor records the brightness of the picture in the scene at the moment of taking pictures or recording, and then restores the color of the scene by means of sub-channel interpolation. However, it is precisely because of the sampling characteristics of the image sensor that the relevant technology ignores the information of the light source in the scene, and the processing of the subsequent algorithm is an irreversible dimensionality reduction process from high-dimensional to low-dimensional, so that the final image and human perception The screens are quite different.

The inventors of this application discovered during the research and development of imaging technology that the human eye’s perception of color is based on the perception of the ambient light spectrum, and the physical basis of the imaging process of electronic equipment should also be the dismantling of the imaging process of the human eye. solution and reconstruction.

However, in related technologies, if spectral information of ambient light needs to be obtained, two sets of devices usually need to be installed, one of which is used to identify the position of the light source in the ambient light, and the other is used to read the spectral information of the position. Before deployment, it is necessary to calibrate and align the two sets of devices, the workload is heavy, and the stability and effect of the finally obtained spectral information are not good.

To sum up, the inventors of the present application found that there are many problems in the sensor-based image acquisition and processing solutions in the related art, so it is necessary to develop a new image acquisition solution to solve the above problems.

The sensor of the embodiment of the present application is described below with reference to FIGS. 1-3 .

As shown in FIG. 1 and FIG. 2 , the sensor according to some embodiments of the present application may be a CMOS (Complementary Metal-Oxide-Semiconductor, Complementary Metal-Oxide-Semiconductor) based image sensor.

The sensor may include a filter unit 10 and a pixel unit 20, and each of the filter unit 10 and the pixel unit 20 may include a plurality, thereby correspondingly forming a filter array and a pixel array. The filter unit 10 and the pixel unit 20 may correspond to the smallest repeating unit in the filter array and the pixel array, or may be a multiple of the smallest repeating unit.

The above-mentioned filter unit 10 may include RGB filters 11 and various narrow-band filters 12. The bandwidths of the various narrow-band filters 12 are continuous within a preset spectral range, and the preset spectral range may include the visible spectrum. Exemplarily, the preset spectral range may be a visible light range of 400nm-700nm.

Wherein, the RGB filter 11 may be a filter required for imaging to obtain RGB image information. Above-mentioned RGB filter 11 can be used for the normal collection of RGB image information, and RGB filter 11 can be color filter, exemplary, RGB filter 11 can comprise red filter R, green filter G and at least one of blue filter B. In some optional examples, the number of green filters G in the RGB filters 11 may be the largest.

The narrow-band filter 12 is a kind of band-pass filter, which can allow optical signals in a specific wavelength band to pass through, while blocking optical signals on both sides that deviate from the specific wavelength band. The half width of the narrow band filter 12 may be smaller than the half width of the RGB filter 11 .

In the example of the present application, various narrow-band filters 12 may refer to multi-spectral narrow-band filters, and various narrow-band filters 12 constitute the channels of multi-spectral signals when the optical path is transmitted, and the number of the multi-spectral signal channels is related to the narrow band The types and quantities of the optical filters 12 are related.

The above-mentioned pixel unit 20 may include a plurality of light-sensitive pixels, and any light-sensitive pixel of the pixel unit 20 may be disposed on one side of the RGB filter 11 or the narrow-band filter 12 of the filter unit 10 .

In some examples, any photosensitive pixel of each pixel unit 20 may be covered with the RGB filter 11 or the narrowband filter 12 in the filter unit 10 .

The aforementioned RGB filter 11 can be used to obtain RGB image information, and various narrow-band filters 12 can be used to obtain multi-spectral information. Exemplarily, when a sensor is used, the first RGB image information can be obtained through the RGB filter 11 , and multi-spectral information corresponding to the first RGB image information can be obtained by using various narrow-band filters 12 . The multispectral information may be a spectral response curve of the object in the first image information.

While using RGB filter 11 to obtain RGB image information, various narrow-band filters 12 set in the sensor can be used to obtain multi-spectral information in the visible light range through fine narrow-band pass, so starting from the imaging principle , improved the structure of the image sensor, and then gave a solution to obtain the ambient light spectrum. The obtained multi-spectral information has a higher dimension than related technologies, and can help guide the digital signal to better restore the shooting environment during subsequent algorithm processing, ensuring the effect of the image. Created for camera photography, dynamic short video It has high use value in commercial applications such as production.

And the entire multi-spectral information capture process can be completed through the sampling control of the image sensor, so there is no need for a complicated field of view calibration and alignment process in the early stage of the project, which greatly simplifies the difficulty of deployment.

In the embodiment of the present application, a filter unit 10 and a pixel unit 20 are arranged in the sensor, wherein the filter unit 10 includes an RGB filter 11 and a variety of narrow-band filters 12, and the photosensitive pixels in the pixel unit 20 and the RGB filter The light sheet 11 or the narrowband filter 12 are set correspondingly, and because a variety of narrowband filters 12 are arranged in the filter unit 10, and the bandwidths of the various narrowband filters 12 are continuous in the preset spectral range, the preset The spectrum includes the visible spectrum, so that the RGB image information can be obtained using the RGB filter 11, and multiple narrow-band filters 12 can obtain multi-spectral information, thereby providing an image data acquisition scheme for obtaining real spectral information in the environment, The light source information in the environmental scene can be collected, so that the subsequent image processing can be based on real spectral information, which indirectly improves the realism of the image and improves the difference between the image and the perception of the human eye.

In some optional examples, along the direction perpendicular to the pixel unit 20 (ie, the X direction in FIG. 1 ), the RGB filter 11 and the narrowband filter 12 may be arranged at intervals. It should be noted that, in the X direction, the RGB filter 11 and the narrow-band filter 12 can be arranged at intervals on the plane perpendicular to the optical path, so that the obtained RGB image information can be more matched with the multi-spectral information, and the calibration degree is high, and the indirect Improved the accuracy of ambient light positions.

Optionally, along the row direction or the column direction (ie, the Y direction in FIG. 1 ) of the plurality of photosensitive pixels, the colors of two adjacent RGB filters 11 of any narrowband filter 12 are different.

Exemplarily, please refer to Figure 1 and Figure 2 together, starting from the upper left corner, the first RGB filter 11 in the first row can be a green narrow-band filter G (that is, the "G" position in Figure 2 ), the second RGB filter 11 can be a red narrow-band filter R (that is, the "R" position in Figure 2), and the second row of RGB filters 11 can be a blue narrow-band filter B in turn ( That is, position "B" in FIG. 2), green narrow-band filter G, or green narrow-band filter G, blue narrow-band filter B, etc. Of course, the RGB filters 11 at different positions can also be replaced with other colors.

In these embodiments, by using the narrow-band filter 12 as a reference object and setting two adjacent RGB filters 11 to have different colors, the restoration degree of RGB image information can be improved.

In some optional examples, the types of two narrow-band filters 12 adjacent to any RGB filter 11 can be different. Exemplarily, Fig. 2 is still used as an example for illustration, when the filter unit 10 is 4* 4. When arranged in a matrix, the RGB filters 11, as shown in the figure, may include a red narrow-band filter R, a blue narrow-band filter B and a green narrow-band filter G. Eight types of narrowband filters 12 may also be included, and each kind of narrowband filter 12 appears only once in the filter unit 10 .

Eight kinds of optical filters constitute 8 channels containing multispectral signals in the optical filter unit 10, which can be respectively recorded as α, β, γ, δ, ε, ζ, η, θ, and the 8 channels include the visible spectrum. Continuous within the preset spectral range, its spectrum design can refer to Figure 3.

The curves of each gray scale in Figure 3 represent the transmittance change of a filter in the visible light range of 400nm-700nm, and the eight channels correspond to eight curves in Figure 3 . These eight curves have different curve positions and are continuous in the wavelength range of 400nm-700nm, thus indicating that the bandwidths of different optical filters are continuous in the visible light range.

In these embodiments, by using the RGB filter 11 as a reference object and setting the types of two adjacent narrow-band filters 12 to be different, the obtained multispectral information can cover the entire visible light range, and the uniformity is better .

In some optional examples, two RGB filters 11 and two kinds of narrowband filters 12 are arranged in a 2*2 matrix, thereby forming a plurality of basic unit blocks 13 in the filter unit 10 .

In the basic unit block 13, two RGB filters 11 are arranged along the first diagonal direction, and the two RGB filters 11 have the same color, and two kinds of narrow-band filters 12 are arranged along the second diagonal direction, and the second RGB filter 11 is arranged along the second diagonal direction. A diagonal direction intersects a second diagonal direction.

Exemplarily, the above-mentioned first diagonal line may be the sub-diagonal line of the basic unit block 13 , and the second diagonal line may be the main diagonal line of the basic unit block 13 . Alternatively, the first diagonal may be the main diagonal of the basic unit block 13 , and the second diagonal may be the sub-diagonal of the basic unit block 13 .

Exemplarily, the first diagonal line and the second diagonal line may also be perpendicular to each other.

It is also required that the first diagonal direction and the second diagonal direction may also be directions parallel to the main and auxiliary diagonals of the basic unit block 13 .

Please continue to refer to Fig. 2, wherein a single filter unit 10 can include four basic unit blocks 13, two kinds of narrow-band filters 12 can be placed in the second diagonal direction of each basic unit block 13, in the first pair Two RGB filters 11 of the same color are placed in the direction of the corner line, and finally arranged in sequence to obtain a filter unit 10 composed of four basic unit blocks 13 .

In these embodiments, by dividing and arranging a plurality of basic unit blocks 13, the arrangement of the filter units 10 is facilitated, the complexity of the process is reduced, and at the same time, the obtained RGB image information can be more matched with the multi-spectral information, and calibration The high accuracy indirectly improves the accuracy of the position of the ambient light source.

Optionally, the colors of the RGB filters 11 in any two adjacent basic unit blocks 13 are different.

Exemplarily, please continue to refer to FIG. 2. In a single filter unit 10, the color of the RGB filter 11 in the basic unit block 13 in the upper left corner is green (that is, the “G” position in FIG. 2 ), and the color of the RGB filter 11 in the upper right corner is green. The color of the RGB filter 11 in the basic unit block 13 is red (with reference to the "R" position in Figure 2), and the color of the RGB filter 11 in the basic unit block 13 in the lower left corner is blue (that is, "B" in Figure 2 " position), the color of the RGB filter 11 in the basic unit block 13 in the lower right corner is green.

Of course, in other examples, the colors of the RGB filters 11 in the basic unit blocks 13 at different positions can also be replaced, as long as the colors of the RGB filters 11 in adjacent basic unit blocks 13 are different.

In these embodiments, the RGB filter 11 in the adjacent basic unit block 13 is set to distinguish colors to help improve the degree of restoration of the first image and ensure the authenticity of the image.

It should also be noted that the multiple narrow-band filters 12 are 8 to 16 narrow-band filters with continuous bandwidths in the visible spectrum. The above-mentioned FIG. 2 shows a schematic distribution diagram of the filter units 10 in the sensor when there are 8 types of narrow-band filters 12 .

In these embodiments, by setting appropriate types of narrow-band filters, it is possible to reduce the complexity of the process as much as possible on the basis of ensuring image restoration and precision, and at the same time ensure an appropriate amount of image data processing.

The sensor of the embodiment of the present application is described in detail above with reference to FIG. 1 to FIG. 3 . On this basis, the embodiment of the present application also protects a camera module and an electronic device, wherein the electronic device may include a casing and a camera module, and the camera module may include the sensor in any of the above examples. Of course, the electronic device may also directly include the sensor in the above embodiment.

The electronic device may be a terminal, or other devices other than the terminal. Exemplarily, the electronic device may be a mobile phone, a tablet computer, a notebook computer, a handheld computer, a vehicle electronic device, a mobile Internet device (MobileInternet Device, MID), an augmented reality (augmented reality, AR)/virtual reality (virtual reality, VR) device, robot, wearable device, ultra-mobile personal computer (ultra-mobile personal computer, UMPC), netbook or personal digital assistant (personal digital assistant, PDA), etc., can also be a server, network attached storage (NetworkAttached Storage, NAS) , a personal computer (personal computer, PC), a television (television, TV), a teller machine, or a self-service machine, etc., which are not specifically limited in this embodiment of the present application.

The electronic device in this embodiment of the present application may be a device with an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in this embodiment of the present application.

The electronic device and the camera module include the sensor provided by the above embodiment, so the electronic device and the camera module have all the beneficial effects of the above sensor, and to avoid repetition, details are not repeated here.

Please refer to FIG. 4, based on the above-mentioned sensor, camera module and electronic equipment, the embodiment of the present application also provides an image processing method. In this embodiment, the method may include:

S410. Obtain the first image and multispectral information corresponding to the first image through the sensor when the shooting instruction is received.

S420. Perform color correction on the first image according to the multispectral information.

It should be noted that the electronic device knows the scene and the spatial distribution of the environment in which it is located from the obtained first image by using the sensor therein, and also obtains the spectral information in the scene by means of various narrow-band filters.

Exemplarily, please refer to Fig. 4 and Fig. 5 together. After the acquisition and processing of the sensor, it can be obtained that there is a tree in the upper left corner of the first image, and the spectral information of this tree can also be obtained (that is, the response on the right side of Fig. 5 curve).

The following is a brief description of the implementation principle of the above scheme:

Assuming that the quantitative sampling interval of the optical filter is 5nm, the imaging model of the sensor can be expressed by formula (1).

Solve the matrix equation, that is, the mixed spectrum information of the environment can be calculated through the grayscale response value of each narrowband channel and the response curve (known value) of each channel of the sensor.

Among them, since the response curve of each channel of the sensor is not necessarily a square matrix, a pseudo-inverse matrix is used.

Combined with the imaging formula, that is, the following formula (2), it can be seen that the spectral information obtained by the sensor is actually a mixed spectrum of the inner product of the light source spectrum I and the object reflection spectrum R at each band. Since the object reflection spectrum R itself is a constant, the result of its inner product with the light source spectrum I will not affect the shape of the light source spectrum I, so the mixed spectrum information in this area is the light source spectrum information. Therefore, image correction processing can be performed directly through the obtained spectral information.

Among them, I is the spectrum of the light source, R is the reflection spectrum of the object, QE is the quantum efficiency sensitivity of the sensor, and Gray _n is the gray value of the nth spectral channel.

It should also be noted that the above image color correction may include white balance processing, color difference correction, etc., and may also include color calibration, image color mode conversion, and application processing of special color effects.

In these examples, after the electronic device collects the multispectral information with the sensor, it can use the multispectral information as the input of the algorithm to help restore the most realistic ambient color of the first image, thereby improving the consistency between human perception and image display .

In some optional examples, the implementation process of the above S420 may include steps A10 to A30.

A10, performing semantic analysis on the first image to obtain a target area, where the target area is an area with a constant reflectance of the object in the first image.

A20, extract target spectral information from the multi-spectral information, where the target spectral information corresponds to the target area.

A30, perform color correction on the target area through the target spectral information.

In these examples, a target area with a constant reflectance of the object in the first image can be obtained through semantic analysis, and the target area is an area that requires white balance processing and color correction. Optionally, on this basis, an area with a constant object reflectance and the highest confidence can also be selected as the target area, thereby improving the accuracy of the target area to be processed.

For example, the target area can be displayed in gray or white. In the actual environment scene, it can be gray concrete floor, white wall, etc. Through multispectral information, the target spectral information of the location of the target area can be obtained, and then according to the target The spectrum information performs image processing on the target area, such as white balance processing, chromatic aberration correction, and 3DLut (3D LookUp Table, 3D color lookup table) processing. As a result, more precise algorithm adaptation is achieved, helping to better restore the sense of reality of the captured first image.

Optionally, as shown in FIG. 6 , the embodiment of the present application also provides an electronic device 600, including a processor 601 and a memory 602, and the memory 602 stores programs or instructions that can run on the processor 601. The When the programs or instructions are executed by the processor 601, the steps of the above-mentioned image processing method embodiments can be realized, and the same technical effect can be achieved, so in order to avoid repetition, details are not repeated here.

It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

FIG. 7 is a schematic diagram of a hardware structure of an electronic device implementing an embodiment of the present application.

The electronic device 700 includes, but is not limited to: a radio frequency unit 701, a network module 702, an audio output unit 703, an input unit 704, a sensor 705, a display unit 706, a user input unit 707, an interface unit 708, a memory 709, and a processor 710, etc. part.

Those skilled in the art can understand that the electronic device 700 can also include a power supply (such as a battery) for supplying power to various components, and the power supply can be logically connected to the processor 710 through the power management system, so that the management of charging, discharging, and function can be realized through the power management system. Consumption management and other functions. The structure of the electronic device shown in FIG. 7 does not constitute a limitation to the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine some components, or arrange different components, and details will not be repeated here. .

The input unit 704 may be configured to obtain the first image and the multispectral information corresponding to the first image through the sensor when the shooting instruction is received.

The processor 710 may be configured to perform color correction on the first image according to the multispectral information.

Optionally, the processor 710 may be configured to perform semantic analysis on the first image to obtain a target area, where the target area is an area with a constant reflectance of an object in the first image; extract target spectral information from multispectral information, and target spectral information Corresponds to the target area; color correction is performed on the target area through the target spectral information.

It should be understood that, in the embodiment of the present application, the input unit 704 may include a graphics processor (Graphics Processing Unit, GPU) 7041 and a microphone 7042, and the graphics processor 7041 is compatible with the image capture device (such as Camera) to process the image data of still pictures or videos. The display unit 706 may include a display panel 7061, and the display panel 7061 may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like. The user input unit 707 includes at least one of a touch panel 7071 and other input devices 7072 . The touch panel 7071 is also called a touch screen. The touch panel 7071 may include two parts, a touch detection device and a touch controller. Other input devices 7072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, switch buttons, etc.), trackballs, mice, and joysticks, which will not be repeated here.

The memory 709 can be used to store software programs as well as various data. The memory 709 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required by at least one function (such as a sound playing function, image playback function, etc.), etc. Furthermore, memory 709 may include volatile memory or nonvolatile memory, or, memory 709 may include both volatile and nonvolatile memory. Wherein, the non-volatile memory may be a read-only memory (Read-Only Memory, ROM), a programmable read-only memory (Programmable ROM, PROM), an erasable programmable read-only memory (Erasable PROM, EPROM), an electronically programmable Erase Programmable Read-Only Memory (Electrically EPROM, EEPROM) or Flash. Volatile memory can be random access memory (Random Access Memory, RAM), static random access memory (Static RAM, SRAM), dynamic random access memory (Dynamic RAM, DRAM), synchronous dynamic random access memory (Synchronous DRAM) , SDRAM), double data rate synchronous dynamic random access memory (Double Data Rate SDRAM, DDRSDRAM), enhanced synchronous dynamic random access memory (Enhanced SDRAM, ESDRAM), synchronous connection dynamic random access memory (Synch link DRAM, SLDRAM) and Direct Memory Bus Random Access Memory (Direct Rambus RAM, DRRAM). The memory 709 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

The processor 710 may include one or more processing units; optionally, the processor 710 integrates an application processor and a modem processor, wherein the application processor mainly handles operations related to the operating system, user interface, and application programs, etc., Modem processors mainly process wireless communication signals, such as baseband processors. It can be understood that the foregoing modem processor may not be integrated into the processor 710 .

In the description of this specification, references to the terms "one embodiment," "some embodiments," "exemplary embodiments," "example," "specific examples," or "some examples" are intended to mean that the implementation A specific feature, structure, material, or characteristic described by an embodiment or example is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

Although the embodiments of the present application have been shown and described, those skilled in the art can understand that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principle and spirit of the present application. The scope of the application is defined by the claims and their equivalents.

Claims (10)

1. A sensor, comprising:

the filter unit comprises RGB filters and a plurality of narrow-band filters, the bandwidths of the plurality of narrow-band filters are continuous in a preset spectral range, and the preset spectral range comprises a visible spectrum;

a pixel unit including a plurality of photosensitive pixels, any one of the photosensitive pixels of the pixel unit being disposed at one side of the RGB filter or the narrow band filter;

the RGB optical filter is used for acquiring RGB image information, and the multiple kinds of narrow-band optical filters are used for acquiring multispectral information.

2. The sensor of claim 1, wherein along a row direction or a column direction of the plurality of photosensitive pixels, two of the RGB filters adjacent to any of the narrowband filters are different in color.

3. The sensor of claim 1, wherein two RGB filters and two narrowband filters are arranged in a 2*2 matrix as a basic unit block;

in the basic unit block, two RGB optical filters are arranged along a first diagonal direction, two narrow-band optical filters are arranged along a second diagonal direction, and the first diagonal direction is intersected with the second diagonal direction;

the two RGB filters in the basic unit block are the same color.

4. The sensor of claim 3, wherein the RGB filters in any two adjacent basic unit blocks are different in color.

5. The sensor of claim 3, wherein the RGB filter comprises one of a red filter, a green filter, and a blue filter.

6. The sensor of claim 1, wherein the plurality of narrowband filters is 8 to 16 narrowband filters having bandwidths that are contiguous in the visible spectral range.

7. A camera module, characterized in that it comprises a sensor according to any one of claims 1 to 6.

8. An electronic device, characterized in that it comprises a sensor according to any one of claims 1 to 6.

9. An image processing method applied to the electronic device according to claim 8, the method comprising:

under the condition that a shooting instruction is received, a first image and multispectral information corresponding to the first image are obtained through the sensor;

and performing color correction on the first image according to the multispectral information.

10. The method according to claim 9, wherein said color correcting said first image based on said multispectral information comprises:

performing semantic analysis on the first image to obtain a target area, wherein the target area is an area with constant object reflectivity in the first image;

extracting target spectrum information from the multispectral information, wherein the target spectrum information corresponds to the target area;

and carrying out color correction on the target area through the target spectrum information.

Images