CN105827754B - A method for generating a high dynamic range image and a mobile terminal - Google Patents

Abstract

The present invention provides the generation method and mobile terminal of a kind of high dynamic range images.This method comprises: obtaining the first camera, the first image, the second image and the third image that second camera and third camera acquire same reference object in synchronization respectively；Image procossing is carried out to the first image, the second image and third image, generates a vertical frame dimension dynamic image；Wherein, the first image is normal exposure image, and the second image is overexposure image, and third image is to owe to expose image.The generation method of high dynamic range images provided by the invention, can be realized user can see the high dynamic range images of reference object in the shooting preview stage in real time, improve the usage experience of user.To solve the problems, such as that existing image pickup method is unable to the high dynamic range images of live preview reference object.

Description

A method for generating a high dynamic range image and a mobile terminal

technical field

Embodiments of the present invention relate to the field of communication technologies, and in particular, to a method for generating a high dynamic range image and a mobile terminal.

Background technique

When using the High-Dynamic Range (HDR) shooting mode, users can get HDR images of their subjects. Compared with ordinary images, HDR images can provide more dynamic range and image details, which are closer to the images seen by the human eye. Therefore, many people like to shoot in HDR mode.

At present, the HDR shooting method is generally as follows: after the user clicks to take a photo, the mobile terminal shoots the shooting object three times in sequence through the same camera, and sequentially obtains one normally exposed image, one overexposed image, and one image of the shooting object. The underexposed image; the HDR image of the subject is then composited from three differently exposed images. Therefore, the user can only see the HDR image of the subject after taking the photo, and cannot preview the HDR image of the subject in real time.

SUMMARY OF THE INVENTION

Embodiments of the present invention provide a method for generating a high dynamic range image and a mobile terminal, so as to solve the problem that the existing shooting method cannot preview the high dynamic range image of the shooting object in real time.

In a first aspect, an embodiment of the present invention provides a method for generating a high dynamic range image, which is applied to a mobile terminal, where the mobile terminal includes a first camera, a second camera, and a third camera, and the method includes:

acquiring a first image, a second image and a third image respectively collected by the first camera, the second camera and the third camera on the same photographing object at the same moment;

Perform image processing on the first image, the second image and the third image to generate a frame of high dynamic range image;

The first image is a normal exposure image, the second image is an overexposed image, and the third image is an underexposed image.

In a second aspect, an embodiment of the present invention further provides a mobile terminal, including a first camera, a second camera, and a third camera, and the mobile terminal further includes:

an image acquisition module, configured to acquire the first image, the second image and the third image respectively collected by the first camera, the second camera and the third camera on the same shooting object;

an image processing module, configured to perform image processing on the first image, the second image and the third image acquired by the image acquisition module to generate a frame of high dynamic range image;

The first image is a normal exposure image, the second image is an overexposed image, and the third image is an underexposed image.

In this way, in this embodiment of the present invention, the first image, the second image and the third image respectively collected by the first camera, the second camera and the third camera for the same shooting object at the same time are obtained; Image processing is performed on the image and the third image to generate a frame of high dynamic range image; wherein the first image is a normal exposure image, the second image is an overexposed image, and the third image is an underexposed image. The user can see the high dynamic range image of the shooting object in real time in the shooting preview stage, which improves the user experience.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present invention more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments of the present invention. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative labor.

1 is a flowchart of a method for generating a high dynamic range image according to a first embodiment of the present invention;

2 is a flowchart of a method for generating a high dynamic range image according to a second embodiment of the present invention;

3A is a schematic structural diagram of a mobile terminal according to a third embodiment of the present invention;

FIG. 3B is another schematic structural diagram of the mobile terminal according to the third embodiment of the present invention;

4 is a schematic structural diagram of a mobile terminal according to a fourth embodiment of the present invention;

FIG. 5 is a schematic structural diagram of a mobile terminal according to a fifth embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

first embodiment

Referring to FIG. 1, FIG. 1 shows a flowchart of a method for generating a high dynamic range image according to the first embodiment of the present invention, and the specific steps are as follows:

Step 101: Acquire a first image, a second image, and a third image respectively collected by the first camera, the second camera, and the third camera on the same photographing object at the same time.

It should be noted that, in step 101, three cameras are used to simultaneously obtain the first image, the second image and the third image of the shooting object. The first image is a normally exposed image, the second image is an overexposed image, and the third image is an underexposed image. By configuring apertures of different sizes for each camera, different exposures of images obtained by each camera can be achieved. It is also possible to set different exposure gain values for each camera to achieve different exposures of images obtained by each camera. Of course, it can be understood that this embodiment does not limit the specific manner in which the exposure of the images obtained by each camera is different.

Step 102: Perform image processing on the first image, the second image and the third image to generate a frame of high dynamic range image.

It should be noted that, step 102 is to obtain a frame of high dynamic range image by synthesizing images with different exposures. Moreover, since three cameras are used in step 101, the first image, the second image and the third image of the shooting object are obtained simultaneously. Therefore, after the processing of step 102, the user can see the high dynamic range image of the shooting object in real time in the shooting preview stage, thereby improving the shooting experience of the user.

In the implementation of the present invention, the first image, the second image and the third image respectively collected by the first camera, the second camera and the third camera for the same shooting object at the same time are obtained; Image processing is performed on the three images to generate a frame of high dynamic range image. The user can see the high dynamic range image of the shooting object in real time in the shooting preview stage, which improves the user experience.

Second Embodiment

Referring to FIG. 2, FIG. 2 shows a flowchart of a method for generating a high dynamic range image according to a second embodiment of the present invention, and the specific steps are as follows:

Step 201: Acquire a first image, a second image and a third image respectively collected by the first camera, the second camera and the third camera on the same shooting object at the same time.

It should be noted that, in step 201, the first image, the second image and the third image of the shooting object are simultaneously obtained through three cameras. The first image is a normally exposed image, the second image is an overexposed image, and the third image is an underexposed image. The way to achieve different exposures of images obtained by each camera may be that the exposure start and end times and exposure gains of the first camera, the second camera and the third camera are the same, and the aperture F value of the second camera is greater than the aperture of the first camera. F value, the aperture F value of the third camera is smaller than the aperture F value of the first camera. For example, the aperture required for normal exposure can be configured for the first camera, that is, the aperture F value of the first camera is the aperture F value required for normal exposure; and the aperture F value of the first camera is 1.414 of the aperture F value of the third camera times, the aperture F value of the second camera is about 1.414 times the aperture F value of the first camera.

It can also be that the first camera, the second camera and the third camera have the same aperture F value and the same exposure start and end times, the exposure gain of the second camera is greater than the exposure gain of the first camera, and the exposure gain of the third camera is smaller than that of the first camera. Exposure gain. For example, the exposure gain of the first camera can be set to the exposure gain required for normal exposure, the exposure gain of the second camera is N times the exposure gain of the first camera, and the exposure gain of the third camera is the exposure gain of the first camera. 1/N times, where N is an integer greater than 1, for example, N is 3. Of course, it can be understood that this embodiment does not limit a specific manner for realizing different exposures of images obtained by each camera.

Step 202: Perform image preprocessing on the first image, the second image, and the third image, respectively, to generate a first intermediate image, a second intermediate image, and a third intermediate image.

It should be noted that the first image, the second image and the third image in step 202 are all images in RAW format, and the image preprocessing process may include: de-mosaic processing, sharpening processing, denoising processing, and color conversion processing and color enhancement processing. The first intermediate image, the second intermediate image and the third intermediate image obtained after image preprocessing are all images in YUV format.

Step 203 , using the first intermediate image as a reference, correct the second intermediate image and the third intermediate image to be the same field of view as the first intermediate image, and obtain the corrected second intermediate image and the third intermediate image.

It should be noted that each camera has a different viewing angle due to a certain distance between the cameras. Therefore, in order to make the images obtained by each camera have the same field of view, it is necessary to adjust the shooting angle of view of each camera according to the angle of view and relative positional relationship of each camera, so that the images captured by each camera have the same field of view. In step 203, by correcting the second intermediate image and the third intermediate image to have the same field of view as the first intermediate image, the images obtained by the three cameras have the same field of view.

Step 204: Extract grayscale information of the first intermediate image, the corrected second intermediate image, and the third intermediate image, respectively, to obtain a first grayscale image, a second grayscale image, and a third grayscale image.

In the embodiment of the present invention, an image in YUV format is divided into three data channels: Y, U, and V, wherein the Y channel represents grayscale information, and the U and V channels represent color information. From the grayscale information of the second intermediate image and the third intermediate image, the data of the Y channel, that is, the grayscale information, can be extracted respectively, and the first grayscale image, the second grayscale image and the third grayscale image can be obtained.

Step 205: Determine the attribute of each pixel in the first grayscale image based on the grayscale value of each pixel in the first grayscale image.

It should be noted that the attributes of each pixel can be divided into normal exposure, underexposure or overexposure. The attribute of each pixel point can be judged based on the gray value of the pixel point. For example, a first threshold value and a second threshold value may be preset. Preferably, the value range of the first threshold value is greater than or equal to 200 and less than 255, and the value range of the second threshold value is greater than 0 and less than 50.

When the grayscale value of the pixel in the first grayscale image is greater than the first threshold, determining that the attribute of the pixel in the first grayscale image is overexposure;

If the grayscale value of the pixel in the first grayscale image is less than the second threshold, determining that the attribute of the pixel in the first grayscale image is underexposure;

If the grayscale value of the pixel in the first grayscale image is greater than or equal to the second threshold and less than or equal to the first threshold, the attribute of the pixel in the first grayscale image is determined to be normal exposure.

Step 206: Assign a value to each pixel in the first grayscale image based on the attribute of each pixel in the first grayscale image.

It should be noted that, in order to obtain a high dynamic range image, after determining the attribute of each pixel in the first grayscale image, it is necessary to determine the first grayscale image according to the attribute of each pixel in the first grayscale image. Each pixel in the image is assigned a gray value. Specifically, the assignment can be performed in the following ways:

If the attribute of the pixel in the first grayscale image is overexposure, the first grayscale value is obtained through the formula G ₁ =g ₁ ×40%+g _w ×60%, and the first gray value is assigned to the first grayscale value. A pixel point in a grayscale image, where _G1 is the first grayscale value, _g1 is the grayscale value of the pixel point in the third grayscale image corresponding to the pixel point in the first grayscale image, _gw is the gray value of the pixel in the first gray image;

If the attribute of the pixel in the first grayscale image is underexposed, the second grayscale value is obtained by formula G ₂ =g ₂ ×40%+g _w ×60%, and the second gray value is assigned to the first grayscale value. A pixel point in a grayscale image, where G2 is the _second grayscale value, g2 is the grayscale value of the pixel point in the _second grayscale image corresponding to the pixel point in the first grayscale image, _gw is the gray value of the pixel in the first gray image;

If the attribute of the pixel in the first grayscale image is normal exposure, the grayscale value of the pixel in the first grayscale image remains unchanged.

Step 207, generating a frame of high dynamic range image.

In this step, after all the pixels whose attributes are under-exposed or over-exposed in the first grayscale image have completed the grayscale value assignment, a fourth intermediate image is generated, and the grayscale of each pixel in the fourth intermediate image is The value is assigned to the pixel at the corresponding position in the first intermediate image, and a frame of high dynamic range image is generated.

In the method for generating a high dynamic range image according to the embodiment of the present invention, the first image, the second image and the third image respectively collected by the first camera, the second camera and the third camera on the same shooting object at the same time are obtained; The first image, the second image and the third image are respectively subjected to image preprocessing to generate the first intermediate image, the second intermediate image and the third intermediate image; Correcting to the same field of view as the first intermediate image, the corrected second intermediate image and the third intermediate image are obtained; respectively extracting the grayscale information of the first intermediate image, the corrected second intermediate image and the third intermediate image, obtaining a first grayscale image, a second grayscale image and a third grayscale image; determining the attribute of each pixel in the first grayscale image based on the grayscale value of each pixel in the first grayscale image; Based on the attributes of each pixel in the first grayscale image, assign a value to each pixel in the first grayscale image; when all the pixels whose attributes are underexposed or overexposed in the first grayscale image are grayscale After the value is assigned, a fourth intermediate image is generated, and the gray value of each pixel in the fourth intermediate image is assigned to a pixel at a corresponding position in the first intermediate image to generate a frame of high dynamic range image. The user can see the high dynamic range image of the shooting object in real time in the shooting preview stage, which improves the user experience.

Third Embodiment

Referring to FIG. 3A, FIG. 3A shows a structural diagram of a mobile terminal according to a third embodiment of the present invention, and the mobile terminal 30 shown in FIG. 3A includes: a first camera 31, a second camera 32, a third camera 33, an image acquisition camera module 34 and image processing module 35, wherein:

The first camera 31, the second camera 32 and the third camera 33 may have the same aperture F value and the same exposure start and end time, the exposure gain of the second camera 32 is greater than that of the first camera 31, and the exposure gain of the third camera 33 is less than The exposure gain of the first camera 31, specifically, the exposure gain of the second camera 32 is N times the exposure gain of the first camera 31, and the exposure gain of the third camera 33 is 1/N times the exposure gain of the first camera 31 , where N is an integer greater than 1.

The exposure start and end time and exposure gain may also be the same, the aperture F value of the second camera 32 is greater than that of the first camera 31 , and the aperture F value of the third camera 33 is smaller than that of the first camera 31 .

The image acquisition module 34 is used to acquire the first image, the second image and the third image respectively collected by the first camera 31, the second camera 32 and the third camera 33 for the same shooting object; wherein the first image is a normal exposure image , the second image is an overexposed image, and the third image is an underexposed image.

The image processing module 35 is configured to perform image processing on the first image, the second image and the third image obtained by the image obtaining module 34 to generate a frame of high dynamic range image.

On the basis of FIG. 3A , optionally, the image processing module 35 includes a preprocessing submodule 351 and a synthesis submodule 352 , and the synthesis submodule 352 includes a correction unit 3521 , an extraction unit 3522 , an attribute determination unit 3523 , and an assignment unit 3524 And the generation unit 3525, the attribute determination unit 3523 includes: a first determination subunit 35231, a second determination subunit 35232 and a third determination subunit 35233, see FIG. 3B, wherein:

The preprocessing sub-module 351 is used to perform image preprocessing on the first image, the second image and the third image acquired by the image acquisition module 304, respectively, to generate the first intermediate image, the second intermediate image and the third intermediate image; wherein, The first image, the second image and the third image are all images in RAW format, and the first intermediate image, the second intermediate image and the third intermediate image are all images in YUV format.

Specifically, the preprocessing sub-module 351 is used for at least one of de-mosaic processing, sharpening processing, denoising processing, color conversion processing and color enhancement processing.

The synthesis sub-module 352 is configured to perform image synthesis processing on the first intermediate image, the second intermediate image and the third intermediate image generated by the preprocessing sub-module 351 according to a preset high dynamic range image synthesis algorithm to generate a frame of high dynamic range image.

The correction unit 3521 is used to correct the second intermediate image and the third intermediate image generated by the preprocessing submodule 351 to be the same field of view as the first intermediate image based on the first intermediate image generated by the preprocessing submodule 351, A corrected second intermediate image and a third intermediate image are obtained.

The extraction unit 3522 is used to extract the grayscale information of the first intermediate image, the second intermediate image and the third intermediate image corrected by the correction unit 3521, respectively, to obtain the first grayscale image, the second grayscale image and the third grayscale image. degree image.

The attribute determination unit 3523 is used to determine the attribute of each pixel in the first grayscale image based on the grayscale value of each pixel in the first grayscale image obtained by the extraction unit 3522, wherein the attributes include normal exposure, Underexposed or overexposed.

The assignment unit 3524 is configured to obtain the first gray value through the formula G ₁ =g ₁ ×40%+g _w ×60% if the attribute determination unit 3523 determines that the attribute of the pixel in the first grayscale image is overexposed , assign the first gray value to a pixel in the first gray image, where G ₁ is the first gray value, and g ₁ is the pixel corresponding to the first gray image in the third gray image The gray value of the pixel point at the position, _gw is the gray value of the pixel point in the first grayscale image;

The assignment unit 3524 is further configured to obtain the second gray value through the formula G ₂ =g ₂ ×40%+g _w ×60% if the attribute determination unit 3523 determines that the attribute of the pixel in the first grayscale image is underexposed , assign the second gray value to the pixel in the first gray image, where G ₂ is the second gray value, and g ₂ is the pixel in the second gray image corresponding to the first gray image The gray value of the pixel point at the position, _gw is the gray value of the pixel point in the first grayscale image;

The assignment unit 3524 is further configured to keep the gray value of the pixel in the first gray image unchanged if the attribute determining unit 3523 determines that the attribute of the pixel in the first grayscale image is normal exposure.

The generating unit 3535 is configured to generate a fourth intermediate image after the assigning unit 3524 completes the gray value assignment for all the pixels whose attributes are under-exposed or over-exposed in the first grayscale image, and assigns each pixel in the fourth intermediate image to the grayscale value. The gray value of each pixel is assigned to the pixel at the corresponding position in the first intermediate image, and a frame of high dynamic range image is generated.

The first determination subunit 35231 is configured to determine that the attribute of the pixel in the first grayscale image is overexposed if the grayscale value of the pixel in the first grayscale image is greater than the first threshold.

The second determination subunit 35232 is configured to determine that the attribute of the pixel in the first grayscale image is underexposed if the grayscale value of the pixel in the first grayscale image is smaller than the second threshold.

The third determination subunit 35233 is configured to determine the attribute of the pixel in the first grayscale image if the grayscale value of the pixel in the first grayscale image is greater than or equal to the second threshold and less than or equal to the first threshold for normal exposure.

The above-mentioned mobile terminal 30 can implement each process implemented by the mobile terminal in the embodiments of FIG. 1 to FIG. 2 , and in order to avoid repetition, details are not repeated here.

In the mobile terminal according to the embodiment of the present invention, the image acquisition module acquires the first image, the second image and the third image respectively collected by the first camera, the second camera and the third camera on the same shooting object at the same time; The first image, the second image and the third image acquired by the image acquisition module are subjected to image processing to generate a frame of high dynamic range image; wherein the first image is a normal exposure image, the second image is an overexposure image, and the third image is underexposed image. The user can see the high dynamic range image of the shooting object in real time in the shooting preview stage, which improves the user experience.

Fourth Embodiment

Referring to FIG. 4, FIG. 4 is a block diagram of a mobile terminal according to a fourth embodiment of the present invention. The mobile terminal 400 shown in FIG. 4 includes: at least one processor 401, memory 402, at least one network interface 403 and other user interfaces 404, and a photographing component 406. The photographing component 406 includes a first camera 4061, a second camera 4062 and a third camera Camera 4063. The various components in the mobile terminal 400 are coupled together by a bus system 405 . It can be understood that the bus system 405 is used to implement the connection communication between these components. In addition to the data bus, the bus system 405 also includes a power bus, a control bus, and a status signal bus. However, for clarity of illustration, the various buses are labeled as bus system 405 in FIG. 4 .

Among others, the user interface 404 may include a display, a keyboard, or a pointing device (eg, a mouse, trackball, touch pad or touch screen, etc.).

It can be understood that the memory 402 in the embodiment of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be Read-Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (ErasablePROM, EPROM), Electrically Erasable Program read-only memory (Electrically EPROM, EEPROM) or flash memory. The volatile memory may be Random Access Memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double data rate synchronous dynamic random access memory (DoubleDataRateSDRAM, DDRSDRAM), enhanced synchronous dynamic random access memory (Enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (SynchlinkDRAM, SLDRAM) and direct memory bus random access Memory (DirectRambusRAM, DRRAM). The memory 402 of the systems and methods described herein is intended to include, but not be limited to, these and any other suitable types of memory.

In some embodiments, memory 402 stores the following elements, executable modules or data structures, or subsets thereof, or extended sets of them: operating system 4021 and applications 4022 .

The operating system 4021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks. The application program 4022 includes various application programs, such as a media player (MediaPlayer), a browser (Browser), etc., for implementing various application services. A program for implementing the method of the embodiment of the present invention may be included in the application program 4022 .

In this embodiment of the present invention, the processor 401 is configured to obtain the first camera 4061 , the second camera 4062 and the third The camera 4063 separately collects the first image, second image and third image of the same shooting object at the same time; image processing is performed on the first image, the second image and the third image to generate a frame of high dynamic range image; wherein, The first image is a normal exposure image, the second image is an overexposed image, and the third image is an underexposed image.

The methods disclosed in the above embodiments of the present invention may be applied to the processor 401 or implemented by the processor 401 . The processor 401 may be an integrated circuit chip with signal processing capability. In the implementation process, each step of the above-mentioned method can be completed by an integrated logic circuit of hardware in the processor 401 or an instruction in the form of software. The above-mentioned processor 401 may be a general-purpose processor, a digital signal processor DSP, an application specific integrated circuit ASIC, an off-the-shelf programmable gate array FPGA or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components. Various methods, steps, and logical block diagrams disclosed in the embodiments of the present invention can be implemented or executed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in conjunction with the embodiments of the present invention may be directly embodied as executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers and other storage media mature in the art. The storage medium is located in the memory 402, and the processor 401 reads the information in the memory 402, and completes the steps of the above method in combination with its hardware.

It will be appreciated that the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (Application Specific Integrated Circuits, ASIC), digital signal processors (Digital Signal Processing, DSP), digital signal processing devices (DSP Device, DSPD), Programmable Logic Device (Programmable Logic Device, PLD) ), a Field-Programmable Gate Array (FPGA), a general-purpose processor, a controller, a microcontroller, a microprocessor, other electronic units for performing the functions described herein, or a combination thereof.

For a software implementation, the techniques described herein may be implemented through modules (eg, procedures, functions, etc.) that perform the functions described herein. Software codes may be stored in memory and executed by a processor. The memory can be implemented in the processor or external to the processor.

Optionally, the aperture F value of the first camera 4061, the second camera 4062 and the third camera 4063 are the same, the exposure start and end times are the same, the exposure gain of the second camera 4062 is greater than the exposure gain of the first camera 4061, and the exposure gain of the third camera 4063 is the same. The exposure gain is smaller than the exposure gain of the first camera 4061 .

Optionally, the exposure start and end time and exposure gain of the first camera 4061, the second camera 4062 and the third camera 4063 are the same, the aperture F value of the second camera 4062 is greater than the aperture F value of the first camera 4061, and the third camera 4063 The aperture F value of is smaller than the aperture F value of the first camera 4061.

Optionally, the processor 401 is further configured to perform image preprocessing on the first image, the second image and the third image respectively, to generate the first intermediate image, the second intermediate image and the third intermediate image; according to the preset high dynamic range The image synthesis algorithm performs image synthesis processing on the first intermediate image, the second intermediate image and the third intermediate image to generate a frame of high dynamic range image; wherein, the first image, the second image and the third image are all in RAW format The images, the first intermediate image, the second intermediate image and the third intermediate image are all images in YUV format.

Optionally, the processor 401 is further configured to use the first intermediate image as a reference, correct the second intermediate image and the third intermediate image to be the same field of view as the first intermediate image, and obtain the corrected second intermediate image and the first intermediate image. Three intermediate images; respectively extract the grayscale information of the first intermediate image, the corrected second intermediate image and the third intermediate image to obtain the first grayscale image, the second grayscale image and the third grayscale image; based on the first grayscale image The gray value of each pixel in the grayscale image determines the attribute of each pixel in the first grayscale image, wherein the attribute includes normal exposure, underexposure or overexposure; if the pixel in the first grayscale image The attribute of the point is overexposure, then the first gray value is obtained by the formula G ₁ =g ₁ ×40%+g _w ×60%, and the first gray value is assigned to the pixel in the first gray image, where , G ₁ is the first gray value, g ₁ is the gray value of the pixel in the third gray image corresponding to the pixel in the first gray image, and g _w is the pixel in the first gray image The gray value of the point; if the attribute of the pixel in the first gray image is underexposed, the second gray value is obtained by the formula G ₂ =g ₂ ×40%+g _w ×60%, and the second gray value is The degree value is assigned to the pixel point in the first grayscale image, wherein G2 is the _second grayscale value, and g2 is the pixel point in the _second grayscale image corresponding to the pixel point in the first grayscale image. Grayscale value, _gw is the grayscale value of the pixel in the first grayscale image; if the attribute of the pixel in the first grayscale image is normal exposure, the grayscale value of the pixel in the first grayscale image remain unchanged; when all the pixels whose attributes are under-exposed or over-exposed in the first grayscale image have completed the grayscale value assignment, a fourth intermediate image is generated, and each pixel in the fourth intermediate image is The gray value of the point is assigned to the pixel point at the corresponding position in the first intermediate image to generate a frame of high dynamic range image.

Optionally, the processor 401 is further configured to determine that the attribute of the pixel in the first grayscale image is overexposure if the grayscale value of the pixel in the first grayscale image is greater than the first preset threshold; The gray value of a pixel in a grayscale image is less than the second preset threshold, then it is determined that the attribute of the pixel in the first grayscale image is underexposed; if the grayscale value of the pixel in the first grayscale image If it is greater than or equal to the second preset threshold and less than or equal to the first preset threshold, it is determined that the attribute of the pixel in the first grayscale image is normal exposure.

Optionally, the processor 401 is further used for at least one of de-mosaic processing, sharpening processing, denoising processing, color conversion processing and color enhancement processing.

Optionally, the exposure gain of the second camera 4062 is N times the exposure gain of the first camera 4061, and the exposure gain of the third camera 4063 is 1/N times the exposure gain of the first camera 4061, where N is greater than 1 the integer.

The mobile terminal 400 can implement each process implemented by the mobile terminal in the foregoing embodiments, and in order to avoid repetition, details are not repeated here.

The mobile terminal according to the embodiment of the present invention obtains the first image, the second image and the third image respectively collected by the first camera, the second camera and the third camera on the same shooting object at the same time; Image processing is performed on the image and the third image to generate a frame of high dynamic range image; wherein the first image is a normal exposure image, the second image is an overexposed image, and the third image is an underexposed image. The user can see the high dynamic range image of the shooting object in real time in the shooting preview stage, which improves the user experience.

Fifth Embodiment

FIG. 5 is a schematic structural diagram of a mobile terminal according to a fifth embodiment of the present invention. Specifically, the mobile terminal 500 in FIG. 5 may be a mobile phone, a tablet computer, a personal digital assistant (Personal Digital Assistant, PDA), or a vehicle-mounted computer.

The terminal 500 in FIG. 5 includes a radio frequency (RF) circuit 501, a memory 502, an input unit 503, a display unit 504, a photographing component 505, a processor 506, an audio circuit 507, a WiFi (Wireless Fidelity) module 508, and a power supply 509 for photographing Assembly 505 includes a first camera 5051 , a second camera 5052 and a third camera 5053 .

Wherein, the input unit 503 can be used for receiving numerical or character information input by the user, and generating signal input related to user setting and function control of the mobile terminal 500 . Specifically, in this embodiment of the present invention, the input unit 503 may include a touch panel 5031 . The touch panel 5031, also known as a touch screen, can collect the user's touch operations on or near it (such as the user's operations on the touch panel 5031 using any suitable objects or accessories such as a finger, a stylus, etc.) The specified program drives the corresponding connection device. Optionally, the touch panel 5031 may include two parts, a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it to the touch controller. To the processor 506, and can receive commands from the processor 506 and execute them. In addition, the touch panel 5031 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch panel 5031, the input unit 503 may also include other input devices 5032, and the other input devices 5032 may include but are not limited to physical keyboards, function keys (such as volume control keys, switch keys, etc.), trackballs, mice, joysticks, etc. one or more of.

The display unit 504 may be used to display information input by the user or information provided to the user and various menu interfaces of the mobile terminal 500 . The display unit 504 may include a display panel 5041, and optionally, the display panel 5041 may be configured in the form of an LCD or an organic light-emitting diode (Organic Light-Emitting Diode, OLED).

It should be noted that the touch panel 5031 can cover the display panel 5041 to form a touch display screen. When the touch display screen detects a touch operation on or near it, it is transmitted to the processor 506 to determine the type of the touch event, and then the processor 506 provides corresponding visual output on the touch display screen according to the type of touch event.

The touch screen includes the application program interface display area and the commonly used controls display area. The arrangement of the application program interface display area and the common control display area is not limited, and may be an arrangement that can distinguish the two display areas, such as up-down arrangement, left-right arrangement, or the like. The application program interface display area can be used to display the interface of the application program. Each interface may contain at least one application icon and/or interface elements such as widget desktop controls. The application program interface display area can also be an empty interface that does not contain any content. The commonly used control display area is used to display controls with high usage rate, such as setting buttons, interface numbers, scroll bars, phonebook icons and other application icons.

The processor 506 is the control center of the mobile terminal 500, using various interfaces and lines to connect various parts of the entire mobile phone, by running or executing the software programs and/or modules stored in the first memory 5021, and calling the software programs and/or modules stored in the second memory 5021. The data in the memory 5022 executes various functions of the mobile terminal 500 and processes data, so as to monitor the mobile terminal 500 as a whole. Optionally, processor 506 may include one or more processing units.

In this embodiment of the present invention, by calling a software program and/or module stored in the first memory 5021 and/or data in the second memory 5022, the processor 506 is configured to acquire the first camera 5051 and the second camera 5052 and the third camera 5053 respectively collect the first image, the second image and the third image of the same object at the same time; image processing is performed on the first image, the second image and the third image to generate a frame of high dynamic range images; wherein, the first image is a normal exposure image, the second image is an overexposed image, and the third image is an underexposed image.

Optionally, the aperture F value of the first camera 5051, the second camera 5052 and the third camera 5053 are the same, the exposure start and end times are the same, the exposure gain of the second camera 5052 is greater than the exposure gain of the first camera 5051, and the exposure gain of the third camera 5053 is the same. The exposure gain is smaller than the exposure gain of the first camera 5051 .

Optionally, the exposure start and end time and exposure gain of the first camera 5051, the second camera 5052 and the third camera 5053 are the same, the aperture F value of the second camera 5052 is greater than the aperture F value of the first camera 5051, and the third camera 5053 The aperture F value of is smaller than the aperture F value of the first camera 5051.

Optionally, the processor 506 is further configured to perform image preprocessing on the first image, the second image and the third image respectively to generate the first intermediate image, the second intermediate image and the third intermediate image; according to the preset high dynamic range The image synthesis algorithm performs image synthesis processing on the first intermediate image, the second intermediate image and the third intermediate image to generate a frame of high dynamic range image; wherein, the first image, the second image and the third image are all in RAW format The images, the first intermediate image, the second intermediate image and the third intermediate image are all images in YUV format.

Optionally, the processor 506 is further configured to use the first intermediate image as a reference, correct the second intermediate image and the third intermediate image to be the same field of view as the first intermediate image, and obtain the corrected second intermediate image and the first intermediate image. Three intermediate images; respectively extract the grayscale information of the first intermediate image, the corrected second intermediate image and the third intermediate image to obtain the first grayscale image, the second grayscale image and the third grayscale image; based on the first grayscale image The gray value of each pixel in the grayscale image determines the attribute of each pixel in the first grayscale image, wherein the attribute includes normal exposure, underexposure or overexposure; if the pixel in the first grayscale image The attribute of the point is overexposure, then the first gray value is obtained by the formula G ₁ =g ₁ ×40%+g _w ×60%, and the first gray value is assigned to the pixel in the first gray image, where , G ₁ is the first gray value, g ₁ is the gray value of the pixel in the third gray image corresponding to the pixel in the first gray image, and g _w is the pixel in the first gray image The gray value of the point; if the attribute of the pixel in the first gray image is underexposed, the second gray value is obtained by the formula G ₂ =g ₂ ×40%+g _w ×60%, and the second gray value is The degree value is assigned to the pixel point in the first grayscale image, wherein G2 is the _second grayscale value, and g2 is the pixel point in the _second grayscale image corresponding to the pixel point in the first grayscale image. Grayscale value, _gw is the grayscale value of the pixel in the first grayscale image; if the attribute of the pixel in the first grayscale image is normal exposure, the grayscale value of the pixel in the first grayscale image remain unchanged; when all the pixels whose attributes are under-exposed or over-exposed in the first grayscale image have completed the grayscale value assignment, a fourth intermediate image is generated, and each pixel in the fourth intermediate image is The gray value of the point is assigned to the pixel point at the corresponding position in the first intermediate image to generate a frame of high dynamic range image.

Optionally, the processor 506 is further configured to determine that the attribute of the pixel in the first grayscale image is overexposure if the grayscale value of the pixel in the first grayscale image is greater than the first preset threshold; The gray value of a pixel in a grayscale image is less than the second preset threshold, then it is determined that the attribute of the pixel in the first grayscale image is underexposed; if the grayscale value of the pixel in the first grayscale image If it is greater than or equal to the second preset threshold and less than or equal to the first preset threshold, it is determined that the attribute of the pixel in the first grayscale image is normal exposure.

Optionally, the processor 506 is further used for at least one of de-mosaic processing, sharpening processing, denoising processing, color conversion processing and color enhancement processing.

Optionally, the exposure gain of the second camera 5052 is N times the exposure gain of the first camera 5051, and the exposure gain of the third camera 5053 is 1/N times the exposure gain of the first camera 5051, where N is greater than 1 the integer.

It can be seen that in this embodiment, the first image, the second image and the third image respectively collected by the first camera, the second camera and the third camera for the same shooting object at the same time are acquired; Image processing is performed on three images to generate a frame of high dynamic range image; wherein, the first image is a normal exposure image, the second image is an overexposed image, and the third image is an underexposed image. The user can see the high dynamic range image of the shooting object in real time in the shooting preview stage, which improves the user experience.

Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of the present invention.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described systems, devices and units may refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

In the embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.

The functions, if implemented in the form of software functional units and sold or used as independent products, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can be embodied in the form of a software product in essence, or the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: a U disk, a removable hard disk, a ROM, a RAM, a magnetic disk, or an optical disk and other mediums that can store program codes.

The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention. should be included within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims (14)

1. A method for generating a high dynamic range image, applied to a mobile terminal, the mobile terminal comprising a first camera, a second camera and a third camera, wherein the method comprises: acquiring a first image, a second image and a third image respectively collected by the first camera, the second camera and the third camera on the same object at the same time; performing image processing on the first image, the second image and the third image to generate a frame of high dynamic range image; Wherein, the first image is a normal exposure image, the second image is an overexposed image, and the third image is an underexposed image; The exposure start and end times of the first camera, the second camera and the third camera are the same, and the acquisition of the first image, the first image and the third camera respectively collected by the first camera, the second camera and the third camera on the same object at the same time. In the step of the second image and the third image, the first image, the second image and the third image of the shooting object are simultaneously obtained through three cameras; The step of performing image processing on the first image, the second image and the third image to generate a frame of high dynamic range image includes: Perform image preprocessing on the first image, the second image and the third image respectively to generate a first intermediate image, a second intermediate image and a third intermediate image; According to a preset high dynamic range image synthesis algorithm, image synthesis processing is performed on the first intermediate image, the second intermediate image and the third intermediate image to generate a frame of high dynamic range image; The step of performing image synthesis processing on the first intermediate image, the second intermediate image and the third intermediate image according to a preset high dynamic range image synthesis algorithm to generate a frame of high dynamic range image includes: Taking the first intermediate image as a benchmark, correcting the second intermediate image and the third intermediate image to be the same field of view as the first intermediate image, to obtain the corrected second intermediate image and the third intermediate image; respectively extracting the grayscale information of the first intermediate image, the corrected second intermediate image and the third intermediate image to obtain the first grayscale image, the second grayscale image and the third grayscale image; determining an attribute of each pixel in the first grayscale image based on the grayscale value of each pixel in the first grayscale image, wherein the attribute includes normal exposure, underexposure, or overexposure; If the attribute of the pixel in the first grayscale image is overexposure, the first grayscale value is obtained by formula G ₁ =g ₁ ×40%+g _w ×60%, and the first grayscale value is is assigned to the pixel point in the first grayscale image, wherein G ₁ is the first grayscale value, and g ₁ is the pixel point in the third grayscale image and the first grayscale image the grayscale value of the pixel point at the corresponding position, and _gw is the grayscale value of the pixel point in the first grayscale image; If the attribute of the pixel in the first grayscale image is underexposure, a second grayscale value is obtained through the formula G ₂ =g ₂ ×40%+g _w ×60%, and the second grayscale value is is assigned to a pixel point in the first grayscale image, wherein G ₂ is the second grayscale value, and g ₂ is the pixel point in the second grayscale image and the first grayscale image the grayscale value of the pixel point at the corresponding position, and _gw is the grayscale value of the pixel point in the first grayscale image; If the attribute of the pixel in the first grayscale image is normal exposure, the grayscale value of the pixel in the first grayscale image remains unchanged; After all the pixels whose attributes are underexposed or overexposed in the first grayscale image have completed the grayscale value assignment, a fourth intermediate image is generated, and the grayscale of each pixel in the fourth intermediate image is The value is assigned to the pixel point at the corresponding position in the first intermediate image to generate a frame of high dynamic range image. 2 . The method according to claim 1 , wherein the aperture F values of the first camera, the second camera and the third camera are the same, and the exposure gain of the second camera is greater than the exposure gain of the first camera. 3 . gain, the exposure gain of the third camera is smaller than the exposure gain of the first camera. 3 . The method according to claim 1 , wherein the exposure gains of the first camera, the second camera and the third camera are the same, and the aperture F value of the second camera is larger than the aperture of the first camera. 4 . F value, the aperture F value of the third camera is smaller than the aperture F value of the first camera. 4 . The method according to claim 1 , wherein the first image, the second image and the third image are all images in RAW format, and the first intermediate image, the second intermediate image and the third intermediate image are in RAW format. 5 . All images are in YUV format. 5 . The method according to claim 1 , wherein, the gray value of each pixel in the first gray image is determined based on the gray value of each pixel in the first gray image. 6 . attribute steps, including: If the grayscale value of the pixel in the first grayscale image is greater than the first preset threshold, determining that the attribute of the pixel in the first grayscale image is overexposure; If the grayscale value of the pixel in the first grayscale image is less than the second preset threshold, determining that the attribute of the pixel in the first grayscale image is underexposure; If the gray value of the pixel in the first grayscale image is greater than or equal to the second preset threshold and less than or equal to the first preset threshold, then determine that the attribute of the pixel in the first grayscale image is Normal exposure. 6. The method according to claim 4, wherein the image preprocessing comprises at least one of de-mosaic processing, sharpening processing, denoising processing, color conversion processing and color enhancement processing. 7 . The method according to claim 2 , wherein the exposure gain of the second camera is N times the exposure gain of the first camera, and the exposure gain of the third camera is the exposure gain of the first camera. 8 . 1/N times the exposure gain of , where N is an integer greater than 1. 8. A mobile terminal comprising a first camera, a second camera and a third camera, wherein the mobile terminal further comprises: an image acquisition module, configured to acquire the first image, the second image and the third image respectively collected by the first camera, the second camera and the third camera on the same shooting object; an image processing module, configured to perform image processing on the first image, the second image and the third image acquired by the image acquisition module to generate a frame of high dynamic range image; Wherein, the first image is a normal exposure image, the second image is an overexposed image, and the third image is an underexposed image; The exposure start and end times of the first camera, the second camera and the third camera are the same, and in the image acquisition module, the first image, the second image and the third image of the object to be photographed are simultaneously obtained through the three cameras; The image processing module includes: a preprocessing submodule, configured to perform image preprocessing on the first image, the second image, and the third image acquired by the image acquisition module, respectively, to generate a first intermediate image, a second intermediate image, and a third intermediate image; The synthesis sub-module is configured to perform image synthesis processing on the first intermediate image, the second intermediate image and the third intermediate image generated by the preprocessing sub-module according to a preset high dynamic range image synthesis algorithm to generate a frame of high dynamic range image; The synthetic submodule includes: A correction unit, configured to correct the second intermediate image and the third intermediate image generated by the preprocessing submodule to be the same as the first intermediate image generated by the preprocessing submodule based on the first intermediate image generated by the preprocessing submodule The intermediate images are in the same field of view, and the corrected second intermediate image and the third intermediate image are obtained; The extraction unit is configured to extract the grayscale information of the first intermediate image, the second intermediate image corrected by the correction unit, and the third intermediate image, respectively, to obtain the first grayscale image, the second grayscale image and the third grayscale image. three grayscale images; an attribute determination unit, configured to determine the attribute of each pixel in the first grayscale image based on the grayscale value of each pixel in the first grayscale image obtained by the extraction unit, wherein the These attributes include normal exposure, underexposure or overexposure; an assignment unit, configured to obtain the first grayscale image by formula G ₁ =g ₁ ×40%+g _w ×60% if the attribute determining unit determines that the attribute of the pixel in the first grayscale image is overexposure The first grayscale value is assigned to the pixel point in the first grayscale image, wherein _G1 is the first grayscale value, and _g1 is the third grayscale image with the grayscale value of the pixel at the corresponding position of the pixel in the first grayscale image, and _gw is the grayscale value of the pixel in the first grayscale image; The assignment unit is further configured to obtain a second gray value by formula G ₂ =g ₂ ×40%+g _w ×60% if the attribute of the pixel in the first grayscale image is underexposed, and The second grayscale value is assigned to a pixel point in the first grayscale image, wherein G ₂ is the second grayscale value, and g ₂ is the difference between the second grayscale image and the first grayscale value. the grayscale value of the pixel at the corresponding position of the pixel in the grayscale image, and _gw is the grayscale value of the pixel in the first grayscale image; The assignment unit is further configured to, if the attribute determination unit determines that the attribute of the pixel in the first grayscale image is normal exposure, the grayscale value of the pixel in the first grayscale image remains unchanged; The generating unit is configured to generate a fourth intermediate image after the assigning unit has completed the assignment of gray values for all the pixels whose attributes are under-exposure or over-exposure in the first gray-scale image. The gray value of each pixel in the image is assigned to the pixel at the corresponding position in the first intermediate image to generate a frame of high dynamic range image. 9 . The mobile terminal according to claim 8 , wherein the aperture F value of the first camera, the second camera and the third camera is the same, and the exposure gain of the second camera is greater than that of the first camera. 10 . Exposure gain, the exposure gain of the third camera is smaller than the exposure gain of the first camera. 10 . The mobile terminal according to claim 8 , wherein the exposure gains of the first camera, the second camera and the third camera are the same, and the aperture F value of the second camera is greater than that of the first camera. 11 . Aperture F value, the aperture F value of the third camera is smaller than the aperture F value of the first camera. 11 . The mobile terminal according to claim 8 , wherein the first image, the second image and the third image are all images in RAW format, and the first intermediate image, the second intermediate image and the third image are in RAW format. 12 . The intermediate images are all images in YUV format. 12. The mobile terminal according to claim 8, wherein the attribute determining unit comprises: a first determination subunit, configured to determine that the attribute of the pixel in the first grayscale image is overexposure if the grayscale value of the pixel in the first grayscale image is greater than a first preset threshold; a second determination subunit, configured to determine that the attribute of the pixel in the first grayscale image is underexposure if the grayscale value of the pixel in the first grayscale image is less than a second preset threshold; a third determination subunit, configured to determine the first grayscale if the grayscale value of the pixel in the first grayscale image is greater than or equal to the second preset threshold and less than or equal to the first preset threshold The properties of the pixels in the image are normal exposure. 13 . The mobile terminal according to claim 11 , wherein the preprocessing submodule is used for at least one of de-mosaic processing, sharpening processing, denoising processing, color conversion processing and color enhancement processing. 14 . 14 . The mobile terminal according to claim 9 , wherein the exposure gain of the second camera is N times the exposure gain of the first camera, and the exposure gain of the third camera is the exposure gain of the first camera. 15 . 1/N times the exposure gain of the camera, where N is an integer greater than 1.