CN109089047B - Method and device for controlling focus, storage medium, and electronic device - Google Patents

Abstract

The application relates to a method and a device for controlling focusing, a storage medium and electronic equipment. And controlling the RGB camera to focus according to the distance from the shooting main body to the TOF camera, and finally shooting the image to be shot by the focused RGB camera to obtain a first target image. The TOF camera is used for simultaneously obtaining the depth data of the whole image by emitting infrared light, and the speed is very high. Therefore, when an image is shot, the distance from the shooting subject to the TOF camera is obtained from the depth data of the image to be shot obtained by the TOF camera very quickly, and the RGB camera is controlled to focus according to the distance from the shooting subject to the TOF camera. And finally, shooting the image to be shot by the focused RGB camera to obtain a first target image. The focusing speed at the time of photographing is finally improved.

Description

Method and device for controlling focus, storage medium, and electronic device

technical field

The present application relates to the field of computer technology, and in particular, to a method and device for controlling focus, a storage medium, and an electronic device.

Background technique

With the rapid development of intelligent mobile terminals, users use mobile terminals with a camera function to take pictures more and more frequently. When a mobile terminal is used to take a picture, a focusing operation is generally required, which may be automatically performed by the camera of the mobile terminal, or may be achieved by the user clicking on the screen. The traditional focusing method has a complicated processing process, resulting in a slow focusing speed.

SUMMARY OF THE INVENTION

The embodiments of the present application provide a method and device for controlling focusing, a storage medium, and an electronic device, which can improve the focusing speed.

A method of controlling focus, including:

Obtain the depth data of the image to be captured through the TOF camera;

Obtain the distance from the subject to the TOF camera from the depth data of the image to be captured;

Control the RGB camera to focus according to the distance from the subject to the TOF camera;

The image to be captured is captured by the focused RGB camera to obtain a first target image.

A device for controlling focusing, the device comprising:

a depth data acquisition module of the image to be shot, used for acquiring the depth data of the image to be shot through the TOF camera;

A distance acquisition module for acquiring the distance from the subject to the TOF camera from the depth data of the image to be shot;

a focusing module, configured to control the RGB camera to focus according to the distance from the shooting subject to the TOF camera;

The photographing module is used for photographing the image to be photographed by the focused RGB camera to obtain a first target image.

A computer-readable storage medium having a computer program stored thereon, the computer program implementing the steps of the above-described method for controlling focus when executed by a processor.

An electronic device includes a memory, a processor, and a computer program stored on the memory and running on the processor. When the processor executes the computer program, the processor executes the steps of the above-mentioned method for controlling focusing.

The above-mentioned method and device for controlling focus, storage medium, and electronic device obtain depth data of the image to be shot through the TOF camera, and obtain the distance from the shooting subject to the TOF camera from the depth data of the image to be shot. The RGB camera is controlled to focus according to the distance from the shooting subject to the TOF camera, and the finally focused RGB camera shoots the to-be-shot image to obtain the first target image. The TOF camera obtains the depth data of the entire image at the same time by emitting infrared light, and the speed is very fast. Therefore, when capturing an image, it is also very fast to obtain the distance from the subject to the TOF camera from the depth data of the image to be captured obtained by the TOF camera, and then control the RGB camera to focus according to the distance from the subject to the TOF camera. The finally focused RGB camera shoots the to-be-shot image to obtain the first target image. So the focus speed when shooting is improved in the end.

Description of drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only These are some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without any creative effort.

1 is an internal structure diagram of an electronic device in one embodiment;

2 is a flowchart of a method for controlling focusing in one embodiment;

3 is a flowchart of a method for obtaining the distance from the subject to the TOF camera from the depth data of the image to be captured in FIG. 2;

4 is a flowchart of a method for obtaining a first focus from an image to be captured in FIG. 3;

5 is a flowchart of a method for controlling focusing in another embodiment;

6 is a schematic structural diagram of a device for controlling focusing in one embodiment;

7 is a schematic structural diagram of a device for controlling focusing in another embodiment;

FIG. 8 is a schematic diagram of an image processing circuit in one embodiment.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application will be described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application.

FIG. 1 is a schematic diagram of the internal structure of an electronic device in one embodiment. As shown in FIG. 1, the electronic device includes a processor, a memory, and a network interface connected through a system bus. Among them, the processor is used to provide computing and control capabilities to support the operation of the entire electronic device. The memory is used for storing data, programs, etc., and at least one computer program is stored in the memory, and the computer program can be executed by the processor to implement the scene recognition method applicable to the electronic device provided in the embodiments of the present application. The memory may include a non-volatile storage medium such as a magnetic disk, an optical disk, and a read-only memory (Read-Only Memory, ROM), or a random-access-memory (Random-Access-Memory, RAM) and the like. For example, in one embodiment, the memory includes a non-volatile storage medium and internal memory. The nonvolatile storage medium stores an operating system and a computer program. The computer program can be executed by the processor to implement a method for controlling focus provided by the following embodiments. Internal memory provides a cached execution environment for operating system computer programs in non-volatile storage media. The network interface can be an Ethernet card or a wireless network card, etc., and is used to communicate with external electronic devices. The electronic device may be a mobile phone, a tablet computer, a personal digital assistant or a wearable device, and the like.

In one embodiment, as shown in FIG. 2, a method for controlling focus is provided, and the method is applied to the electronic device in FIG. 1 as an example for description, including:

Step 220: Acquire depth data of the image to be shot through the TOF camera.

TOF is the abbreviation of Time of flight, which literally means time of flight. TOF cameras are time-of-flight cameras. The TOF camera continuously sends light pulses to the target, and then uses the sensor to receive the light returned from the object, and obtains the distance of the target object by detecting the flight (round-trip) time of the light pulse. This technology is different in principle from the 3D laser sensor. The 3D laser sensor scans point by point, while the TOF camera obtains the depth information of the entire image at the same time. The image to be captured here may refer to a preview image presented on the display screen of the electronic device when the camera is turned on.

Step 240: Obtain the distance from the shooting subject to the TOF camera from the depth data of the image to be shot.

Usually when shooting, there will be a subject. When autofocusing, the camera generally selects the object closest to the lens and with the greatest contrast as the subject, and it often happens that the subject the camera automatically finds is not the one the user needs, or the position of the object to be photographed does not meet the user's expectations. Composition, then you must use manual selection of the subject. For example, when the foreground of the image to be shot is a person, and the background is a blue sky and white clouds, the camera can automatically take the person as the subject of shooting. Of course, the person in the preview image can also be taken as the subject by manually clicking on the screen by the user. However, when the proportion of the background in the image to be shot is very small, and the proportion of the image included in the foreground is very high, and the number of people is relatively large, it is generally difficult to automatically determine the subject by the camera. Take one or several people in the preview image as the subject by tapping the screen.

After the shooting subject is determined, the distance from the shooting subject to the TOF camera is obtained from the depth data of the image to be shot. Specifically, the first focus is acquired from the image to be shot, and the first focus is located on the shooting subject. The depth data corresponding to the first focus is obtained from the depth data of the image to be shot, and the depth data corresponding to the first focus is used as the distance from the shooting subject to the TOF camera.

Step 260, controlling the RGB camera to focus according to the distance from the shooting subject to the TOF camera.

After the distance from the subject to the TOF camera is obtained, the motor parameters of the RGB camera are calculated according to the distance from the subject to the TOF camera, and then the motor of the RGB camera is controlled to move according to the calculated motor parameters of the RGB camera to achieve focusing. The number of RGB cameras on an electronic device is not limited.

Step 280, the image to be captured is captured by the focused RGB camera to obtain a first target image.

The depth data from the first focus on the shooting subject to the TOF camera (which may also be an imaging plane) collected by the TOF camera is used, and the RGB camera is controlled to focus according to the depth data. Specifically, according to the depth data, the calibration data that the RGB camera and the TOF camera are calibrated at the same time, and the motor parameters of the RGB camera in the current camera, the depth data from the first focus to the TOF camera is converted into the motor of the RGB camera trying to focus. Move steps. Then, the motor of the RGB camera is moved by the corresponding number of steps, and the focusing according to the first focus can be realized. Then, the image to be shot is captured by the focused RGB camera to obtain the first target image.

In the embodiment of the present application, compared with a binocular stereo camera or a triangulation system, the TOF camera is small in size, which is similar to the size of a general camera, and is very suitable for some occasions that require a light and small-sized camera. TOF cameras can quickly calculate depth information in real time, reaching tens to 100fps. Secondly, the binocular stereo camera needs to use a complex correlation algorithm, and the processing speed is slow. The depth calculation accuracy of TOF does not change with the change of distance, and can basically be stabilized at the cm level, which is very meaningful for some large-scale motion applications. In addition, the TOF camera can still measure the distance in the dark environment, which solves the problem of out-of-focus of the RGB camera in the dark environment.

The depth data of the image to be shot is acquired through the TOF camera, and the distance from the subject to the TOF camera is acquired from the depth data of the image to be shot. Control the RGB camera to focus according to the distance from the subject to the TOF camera. The TOF camera obtains the depth data of the entire image at the same time by emitting infrared light, and the speed is very fast. Therefore, when capturing an image, it is also very fast to obtain the distance from the subject to the TOF camera from the depth data of the image to be captured obtained by the TOF camera, and then control the RGB camera to focus according to the distance from the subject to the TOF camera. Therefore, the focusing speed during shooting is finally improved, and the focusing accuracy is higher, and accurate focusing can also be performed in a dark light environment. Finally, the image to be captured is captured by the focused RGB camera to obtain a first target image. Obviously, the image quality of the first target image captured after the above focusing process will be greatly improved.

In one embodiment, as shown in FIG. 3, step 240, obtaining the distance from the shooting subject to the TOF camera from the depth data of the image to be shot, including:

Step 242: Acquire a first focus from the image to be shot, where the first focus is on the shooting subject.

First, the subject needs to be determined from the image to be captured, and then the first focus needs to be determined from the subject. The above steps of determining the subject and determining the first focus can be automatically set by the camera, and of course, can also be determined by the user manually clicking on the preview image appearing on the electronic screen. The camera generally chooses to automatically select the object closest to the lens with the greatest contrast as the subject, and select a certain point on the subject as the first focus. For example, when the foreground of the image to be shot is a person and the background is a blue sky and white clouds, the camera can automatically take the foreground person who is closest to the lens and has the greatest contrast as the subject. Then select a certain point (eg, the bridge of the nose) from the face part of the foreground person as the first focus.

When the shooting subject and the first focus automatically determined by the camera do not meet the user's shooting requirements, the user can manually click the preview image that appears on the electronic screen to determine the shooting subject, and then determine the first focus from the shooting subject. .

Step 244: Acquire depth data corresponding to the first focus from the depth data of the image to be captured.

After the shooting subject and the first focus are determined, the depth data corresponding to the first focus can be acquired from the depth data of the image to be shot. The depth data is the depth data of the image to be shot acquired by the TOF camera emitting and receiving infrared light. The acquisition process is not only fast and accurate, but also suitable for bright and dark environments.

Step 246, taking the depth data corresponding to the first focus as the distance from the shooting subject to the TOF camera.

The distance from the subject to the TOF camera can be understood as the focal length. The depth data corresponding to the first focus is taken as the distance from the subject to the TOF camera. Subsequently, the RGB camera can be controlled to focus according to the distance from the subject to the TOF camera.

In the embodiment of the present application, after the subject is determined from the image to be captured, and the first focus is further determined from the subject. The depth data of the first focus is correspondingly obtained from the depth data of the to-be-shot image obtained directly from the TOF camera. The depth data of the first focus is the distance from the subject to the TOF camera. From the depth data of the image to be captured obtained by the TOF camera, the accuracy of the obtained distance from the subject to the TOF camera is very high. Therefore, the accuracy of controlling the focus of the RGB camera according to the obtained distance from the subject to the TOF camera is further improved.

In one embodiment, as shown in FIG. 4 , step 242 , acquiring a first focus from the image to be shot, where the first focus is located on the shooting subject, includes:

Step 242a, acquiring the subject from the image to be shot;

Step 242b, determining a plurality of focal points to be selected from the shooting subject, and the plurality of focal points to be selected have different depth data;

Step 242c, respectively performing a shooting preview according to each focus to be selected to obtain a plurality of preview images;

Step 242d: Screen out the target preview image from the plurality of preview images, obtain the focus to be selected corresponding to the target preview image, and use the focus to be selected as the first focus.

Specifically, after the subject is acquired from the image to be shot according to the method in the above embodiment, a plurality of foci to be selected are determined from the subject, and the foci to be selected have different depth data respectively. The depth data of the multiple to-be-selected focal points may be determined according to the arithmetic sequence. For example, when it is determined from the image to be photographed that the subject is a person, it is assumed that there are multiple persons in the image to be photographed, and the multiple persons are standing together one after the other. At this point, multiple characters have different depth data. The depth data is filtered from the depth data of these characters according to the order of the arithmetic sequence, that is, the filtered depth data is arranged in an arithmetic sequence. For example, filter a data from the depth data of the first person standing closest to the camera (for example, 3 meters), and then from the distance of the second person next to the first person from the camera. One piece of data (for example, 3.05 meters is selected from the depth data, at this time, the tolerance of the arithmetic sequence is 0.05 meters, of course, other values can be set in other embodiments), and so on to obtain a set of arithmetic sequences. The focal points corresponding to the depth values in the arithmetic progression are the focal points to be selected.

Shooting preview is performed according to each focus to be selected, and multiple preview images are obtained. For example, to shoot the to-be-shot image according to the focus to be selected, that is, to adjust the focus of the RGB camera according to the depth values in the arithmetic sequence, so as to obtain a set of preview images respectively. The shooting focus corresponding to each preview image is different, so the image quality of each preview image will be different. The image quality here may include sharpness, brightness, chromaticity, etc., which are not limited here.

After a group of preview images is obtained, a target preview image is selected from the plurality of preview images, and the target preview image is a preview image with the best image quality in the group of preview images. Screening out the preview image with the best image quality from multiple preview images can be done automatically by the processor inside the camera. After a preview image with the best image quality is selected, a focus to be selected corresponding to the preview image of the target is acquired, and the focus to be selected is used as the first focus. Finally, when the image to be processed is formally photographed, the depth data of the first focus can be obtained according to the first focus, and the depth data corresponding to the first focus can be used as the distance from the shooting subject to the TOF camera. The RGB camera is controlled to focus according to the distance from the shooting subject to the TOF camera, and the focused RGB camera shoots the image to be shot to obtain the first target image.

In the embodiment of the present application, when the first focus is determined from the image to be processed, a plurality of focus points to be selected are firstly determined from the shooting subject, and each focus point to be selected has different depth data. Thus, a group of preview images are captured according to the multiple focal points to be selected, and these preview images are captured according to the focal points with different depth data respectively. Therefore, the image with the best image quality is selected from the plurality of preview images, so that the result after the comparison and screening is more accurate, which is more conducive to improving the accuracy of the determined first focus, thereby improving the image captured according to the first focus. The image quality of the first target image. And the depth data is obtained by the TOF camera. The TOF camera obtains the depth data of the entire image at the same time by emitting infrared light, and the speed is very fast. Therefore, while improving the focusing speed, the image quality of the captured first target image is also guaranteed.

In one embodiment, as shown in FIG. 5 , in step 280, the image to be captured is captured by the focused RGB camera to obtain the first target image, including:

Step 510: Determine whether the image quality of the first target image reaches a preset standard.

The preset standard may be pre-stored in the electronic device, for example, the definition of the image may reach a certain standard, or the color and brightness of the image may reach a certain standard.

Step 520, if not, acquire a point within a preset range centered on the first focus from the image to be shot as the second focus, and the second focus is located on the shooting subject.

When the first focus is preliminarily determined, and the RGB camera is controlled to focus according to the obtained distance from the subject to the TOF camera, and the first target image captured is judged whether the image quality reaches the preset standard, if the judgment result is When the preset standard is reached, there is no need to re-determine the second focus.

If the judgment result is that the preset standard is not met, the second focus needs to be re-determined again. Of course, the second focus is still on the shooting subject. A point within a preset range centered on the first focus is acquired from the image to be shot as the second focus. For example, it may be to move a specific number of pixels to the left or right, or up or down, respectively, within a preset range centered on the first focus, so that the acquired pixels correspond to the points on the subject as Second focus. Specifically, which direction to move and how many pixels to move can be determined according to the image quality of the first target image. For example, when the clarity of the first target image is lower than the preset standard, it means that the determined depth data of the first focus is greater than the focal length when the shooting is clear. Therefore, when the second focus is determined, the A point near a focal point whose depth data is slightly smaller than that of the first focal point is selected as the second focal point. Assuming that the depth data of the first focus is 4 meters, a point near the first focus and the depth data of 3.95 meters is selected as the second focus with an interval of 0.05 meters. Of course, this interval can also be set to other reasonable values.

When the clarity of the first target image meets the preset standard, but the brightness is lower than the preset standard, it means that the determined depth data of the first focus meets the preset standard, but the brightness at the first focus is relatively bright, will result in a darker entire image. At this time, in order to make the brightness of the image meet the preset standard, the depth data near the first focus can be selected to be the same as the depth data of the first focus, but the brightness is darker as the second focus. The brightness of the image captured by the difocal point will be improved.

Step 530: Acquire depth data corresponding to the second focus from the depth data of the image to be captured.

Step 540, taking the depth data corresponding to the second focus as the distance from the shooting subject to the TOF camera.

According to the position of the second focus on the image to be captured, the depth data corresponding to the second focus is acquired from the depth data of the image to be captured. The depth data corresponding to the second focus is taken as the distance from the subject to the TOF camera. It can be considered that the depth data corresponding to the second focus is the focal length.

Step 550: Control the RGB camera to focus according to the distance from the shooting subject to the TOF camera.

Step 560, the image to be captured is captured by the focused RGB camera to obtain a second target image.

According to the depth data corresponding to the second focus, the calibration data that the RGB camera and the TOF camera are calibrated at the same time, and the motor parameters of the RGB camera in the current camera, convert the depth data from the second focus to the TOF camera into the motor of the RGB camera. The number of steps to move the focus. Then, the motor of the RGB camera is moved by the corresponding number of steps, and the focusing can be achieved according to the second focus. Then, the to-be-shot image is captured by the refocused RGB camera to obtain a second target image.

In the embodiment of the present application, after the depth data of the first focus is obtained for the first time according to the depth data collected by the TOF camera corresponding to the first focus roughly obtained from the shooting subject. The RGB camera is focused according to the depth data of the first focus, and the first target image is obtained by using the focused RGB camera to shoot. If it is determined that the image quality of the first target image does not meet the preset standard, it is necessary to fine-tune the focusing effect. You can try to move a specific number of pixels to the left or right or up or down within the preset range centered on the first focus, so that the acquired pixels correspond to the points on the subject as the second. focus. Similarly, focus the RGB camera according to the depth data of the second focus, and use the focused RGB camera to shoot to obtain a second target image. It is judged again whether the image quality of the second target image meets the preset standard. This is done until the resulting image quality meets the preset standard. After many attempts to change the focus position, the inaccuracy of the first focus obtained at one time can be avoided, resulting in that although the depth data of the first focus can be accurately obtained according to the TOF camera, it still cannot be obtained from the source (that is, the focus position). ) to improve the quality of the captured image.

In one embodiment, controlling the RGB camera to focus according to the distance from the subject to the TOF camera includes:

Calculate the motor parameters of the RGB camera according to the distance from the subject to the TOF camera;

The motor of the RGB camera is controlled to move according to the calculated motor parameters of the RGB camera to achieve focusing.

In the embodiment of the present application, according to the distance from the shooting subject to the TOF camera, the calibration data that the RGB camera and the TOF camera are calibrated at the same time, and the motor parameters of the RGB camera in the current camera, the first focus on the shooting subject is set to the TOF camera. The depth data is converted into the number of steps that the RGB camera's motor tries to focus on. Then, the motor of the RGB camera is moved by the corresponding number of steps, and the focusing according to the first focus can be realized. According to the depth data of the TOF camera, the motor of the RGB camera can be moved quickly and accurately to achieve focusing.

In one embodiment, the depth data of the image to be captured is obtained by the TOF camera, including:

Obtain multiple frames of TOF data of different phases of the image to be captured through the TOF camera, and each phase corresponds to one frame of TOF data;

Multiple frames of TOF data are synthesized to obtain depth data of the image to be captured.

In the embodiment of the present application, a frame of depth data of an image to be captured is generally synthesized by a frame of TOF data corresponding to 4 phases or 8 phases respectively. When the selected phase is 4, a frame of TOF data corresponding to the 4 phases of the image to be captured is obtained through the TOF camera, and there are 4 frames of TOF data. The 4 frames of TOF data are synthesized to obtain the depth data of the image to be captured. Of course, when the selected phase is 8, a frame of TOF data corresponding to the 8 phases of the image to be captured is obtained through the TOF camera, and there are 8 frames of TOF data. The 8 frames of TOF data are synthesized to obtain the depth data of the image to be captured.

In one embodiment, after acquiring the depth data of the image to be captured by the TOF camera, it includes:

Filter and denoise the depth data of the image to be captured.

In the embodiment of the present application, Poisson equation filtering, Gaussian filtering and bilateral filtering methods may be used to perform filtering and denoising operations on the depth data of the image to be captured. Therefore, the depth data of the to-be-shot image obtained after filtering and denoising processing is more accurate.

In one embodiment, as shown in FIG. 6 , an apparatus 600 for controlling focusing is provided, including: a depth data acquisition module 610 , a distance acquisition module 620 , a focusing module 630 and a photographing module 640 of an image to be captured. in,

The depth data acquisition module 610 of the image to be shot is used to acquire the depth data of the image to be shot through the TOF camera;

A distance acquisition module 620, configured to acquire the distance from the shooting subject to the TOF camera from the depth data of the image to be shot;

The focusing module 630 is used to control the RGB camera to focus according to the distance from the shooting subject to the TOF camera;

The photographing module 640 is used for photographing the image to be photographed by the focused RGB camera to obtain the first target image.

In one embodiment, the distance acquisition module 620 is further configured to acquire a first focus from the image to be shot, where the first focus is located on the subject; acquire depth data corresponding to the first focus from the depth data of the image to be shot; The depth data corresponding to the first focus is taken as the distance from the subject to the TOF camera.

In one embodiment, the distance acquisition module 620 is further configured to acquire the subject from the image to be shot; determine multiple focal points to be selected from the subject to be shot, and the multiple focal points to be selected have different depth data; Selecting a focus for shooting and previewing to obtain a plurality of preview images; screening out a target preview image from the plurality of preview images, obtaining a focus to be selected corresponding to the target preview image, and taking the focus to be selected as the first focus.

In one embodiment, as shown in FIG. 7 , an apparatus 600 for controlling focusing is provided, further comprising:

A judgment module 650, configured to judge whether the image quality of the first target image reaches a preset standard;

The second focus acquisition module 660 is configured to, if not, acquire a point within a preset range centered on the first focus from the image to be shot as the second focus, and the second focus is located on the shooting subject;

The refocusing module 670 is used to obtain the depth data corresponding to the second focus from the depth data of the image to be shot; the depth data corresponding to the second focus is used as the distance from the subject to the TOF camera; according to the distance from the subject to the TOF camera Control the RGB camera to focus;

The re-shooting module 680 is used for capturing the image to be captured by the focused RGB camera to obtain a second target image.

In one embodiment, the focusing module 630 is further configured to calculate the motor parameters of the RGB camera according to the distance from the shooting subject to the TOF camera; and control the motor movement of the RGB camera according to the calculated motor parameters of the RGB camera to realize focusing.

In one embodiment, the depth data acquisition module 610 of the image to be shot is further configured to obtain multiple frames of TOF data of different phases of the image to be shot through the TOF camera, and each phase corresponds to one frame of TOF data; Perform synthesis to obtain depth data of the image to be captured.

In one embodiment, the depth data acquisition module 610 of the image to be captured is further configured to perform filtering and denoising operations on the depth data of the image to be captured.

The division of each module in the above apparatus for focusing control is only for illustration. In other embodiments, the apparatus for controlling focusing may be divided into different modules as required to complete all or part of the functions of the apparatus for controlling focusing.

In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, implements the steps of the methods for controlling focus provided by the foregoing embodiments.

In one embodiment, an electronic device is provided, which includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the control focusing provided by the above embodiments is implemented. steps of the method.

Embodiments of the present application also provide a computer program product, which, when running on a computer, causes the computer to execute the steps of the methods for controlling focus provided by the above embodiments.

The embodiments of the present application also provide an electronic device. The electronic device may be any terminal device including a mobile phone, a tablet computer, a PDA (Personal Digital Assistant), a POS (Point of Sales, a sales terminal), a vehicle-mounted computer, a wearable device, etc. The electronic device is a mobile phone as an example : The above electronic device includes an image processing circuit, and the image processing circuit may be implemented by hardware and/or software components, and may include various processing units that define an ISP (Image Signal Processing, image signal processing) pipeline. FIG. 8 is a schematic diagram of an image processing circuit in one embodiment. As shown in FIG. 8 , for the convenience of description, only various aspects of the image processing technology related to the embodiments of the present application are shown.

As shown in FIG. 8 , the image processing circuit includes a first ISP processor 830 , a second ISP processor 840 and a control logic 850 . The first camera 810 includes one or more first lenses 812 and a first image sensor 814 . The first image sensor 814 may include a color filter array (eg, a Bayer filter), the first image sensor 814 may obtain light intensity and wavelength information captured with each imaging pixel of the first image sensor 814, and provide information that can be accessed by the first ISP A set of image data processed by processor 830. The second camera 820 includes one or more second lenses 822 and a second image sensor 824 . The second image sensor 824 may include a color filter array (eg, a Bayer filter), the second image sensor 824 may acquire light intensity and wavelength information captured with each imaging pixel of the second image sensor 824, and provide information that can be accessed by the second ISP A set of image data processed by processor 840 .

The first image captured by the first camera 810 is transmitted to the first ISP processor 830 for processing. After the first ISP processor 830 processes the first image, the statistical data of the first image (such as the brightness of the image, the contrast value of the image, etc.) , image color, etc.) to the control logic 850, and the control logic 850 can determine the control parameters of the first camera 810 according to the statistical data, so that the first camera 810 can perform auto-focus, auto-exposure and other operations according to the control parameters. After being processed by the first ISP processor 830, the first image can be stored in the image memory 860, and the first ISP processor 830 can also read the image stored in the image memory 860 for processing. In addition, after being processed by the ISP processor 830, the first image can be directly sent to the display 870 for display, and the display 870 can also read the image in the image memory 860 for display.

Among them, the first ISP processor 830 processes the image data pixel by pixel in various formats. For example, each image pixel may have a bit depth of 8, 10, 12, or 14 bits, and the first ISP processor 830 may perform one or more image processing operations on the image data, collecting statistical information about the image data. Among them, the image processing operations can be performed with the same or different bit depth calculation precision.

The image memory 860 may be a part of a memory device, a storage device, or an independent dedicated memory within an electronic device, and may include a DMA (Direct Memory Access, direct memory access) feature.

Upon receiving the interface from the first image sensor 814, the first ISP processor 830 may perform one or more image processing operations, such as temporal filtering. The processed image data may be sent to image memory 860 for additional processing before being displayed. The first ISP processor 830 receives processing data from the image memory 860, and performs image data processing in RGB and YCbCr color spaces on the processed data. The image data processed by the first ISP processor 830 may be output to the display 870 for viewing by the user and/or further processed by a graphics engine or a GPU (Graphics Processing Unit, graphics processor). In addition, the output of the first ISP processor 830 may also be sent to the image memory 860 , and the display 870 may read image data from the image memory 860 . In one embodiment, image memory 860 may be configured to implement one or more frame buffers.

Statistics determined by the first ISP processor 830 may be sent to the control logic 850 . For example, the statistics may include first image sensor 814 statistics such as auto exposure, auto white balance, auto focus, flicker detection, black level compensation, first lens 812 shading correction, and the like. The control logic 850 may include a processor and/or a microcontroller executing one or more routines (eg, firmware) that may determine the control parameters and the first camera 810 based on the received statistical data. A control parameter of the ISP processor 830. For example, the control parameters of the first camera 810 may include gain, exposure control integration time, anti-shake parameters, flash control parameters, first lens 812 control parameters (eg, focal length for focusing or zooming), or a combination of these parameters. ISP control parameters may include gain levels and color correction matrices for automatic white balance and color adjustment (eg, during RGB processing), and first lens 812 shading correction parameters.

Similarly, the second image captured by the second camera 820 is transmitted to the second ISP processor 840 for processing. After the second ISP processor 840 processes the first image, the statistical data of the second image (such as the brightness of the image, the The contrast value, the color of the image, etc.) are sent to the control logic 850, and the control logic 850 can determine the control parameters of the second camera 820 according to the statistical data, so that the second camera 820 can perform auto-focus, auto-exposure and other operations according to the control parameters . The second image can be stored in the image memory 860 after being processed by the second ISP processor 840, and the second ISP processor 840 can also read the image stored in the image memory 860 for processing. In addition, after being processed by the ISP processor 840, the second image can be directly sent to the display 870 for display, and the display 870 can also read the image in the image memory 860 for display. The second camera 820 and the second ISP processor 840 can also implement the processing procedures as described for the first camera 810 and the first ISP processor 830 .

The following are steps for implementing the image processing method using the image processing technology in FIG. 8 .

Any reference to a memory, storage, database, or other medium as used herein may include non-volatile and/or volatile memory. Suitable nonvolatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which acts as external cache memory. By way of illustration and not limitation, RAM is available in various forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous Link (Synchlink) DRAM (SLDRAM), Memory Bus (Rambus) Direct RAM (RDRAM), Direct Memory Bus Dynamic RAM (DRDRAM), and Memory Bus Dynamic RAM (RDRAM).

The above-mentioned embodiments only represent several embodiments of the present application, and the descriptions thereof are relatively specific and detailed, but should not be construed as a limitation on the scope of the patent of the present application. It should be pointed out that for those skilled in the art, without departing from the concept of the present application, several modifications and improvements can be made, which all belong to the protection scope of the present application. Therefore, the scope of protection of the patent of the present application shall be subject to the appended claims.

Claims (14)

1. a method for controlling focus, is characterized in that, comprises: Obtain the depth data of the image to be captured through the TOF camera; Obtain the distance from the subject to the TOF camera from the depth data of the image to be captured; Control the RGB camera to focus according to the distance from the subject to the TOF camera; The image to be captured is captured by the focused RGB camera to obtain a first target image; Determine whether the image quality of the first target image reaches a preset standard; the image quality includes clarity and brightness; If not, based on the comparison result of the different quality parameters of the first target image and the preset standard, a point within a preset range centered on the first focus is acquired in the to-be-shot image as the second focus. the focus and the second focus are located on the subject; acquiring depth data corresponding to the second focus from the depth data of the to-be-shot image; Taking the depth data corresponding to the second focus as the distance from the shooting subject to the TOF camera; Control the RGB camera to focus according to the distance from the subject to the TOF camera; The image to be captured is captured by the focused RGB camera to obtain a second target image. 2. The method according to claim 1, wherein the obtaining the distance from the subject to the TOF camera from the depth data of the to-be-shot image comprises: obtaining a first focus from the to-be-shot image; acquiring depth data corresponding to the first focus from the depth data of the to-be-shot image; The depth data corresponding to the first focus is taken as the distance from the shooting subject to the TOF camera. 3 . The method according to claim 2 , wherein the acquiring a first focus from the to-be-shot image, the first focus being located on a shooting subject, comprises: 3 . acquiring the subject from the to-be-shot image; Determine a plurality of to-be-selected focal points from the photographic subject, and the plurality of to-be-selected focal points have different depth data; Shooting and previewing according to each of the to-be-selected focal points, respectively, to obtain a plurality of preview images; A target preview image is selected from the plurality of preview images, a focus to be selected corresponding to the target preview image is acquired, and the focus to be selected is used as the first focus. 4. The method according to claim 1, wherein the controlling the RGB camera to focus according to the distance from the shooting subject to the TOF camera, comprising: Calculate the motor parameters of the RGB camera according to the distance from the shooting subject to the TOF camera; The motor of the RGB camera is controlled to move according to the calculated motor parameters of the RGB camera, so as to realize focusing. 5. The method according to claim 1, wherein the obtaining the depth data of the image to be captured by the TOF camera comprises: Obtain multiple frames of TOF data of different phases of the image to be captured through the TOF camera, and each phase corresponds to one frame of TOF data; The multiple frames of TOF data are synthesized to obtain the depth data of the image to be captured. 6. The method according to claim 1, characterized in that, after obtaining the depth data of the image to be captured by the TOF camera, comprising: Filtering and denoising operations are performed on the depth data of the to-be-shot image. 7. A device for controlling focusing, wherein the device comprises: a depth data acquisition module of the image to be shot, used for acquiring the depth data of the image to be shot through the TOF camera; A distance acquisition module for acquiring the distance from the subject to the TOF camera from the depth data of the image to be shot; a focusing module, configured to control the RGB camera to focus according to the distance from the shooting subject to the TOF camera; a shooting module, used for shooting the image to be shot by the focused RGB camera to obtain a first target image; a judgment module for judging whether the image quality of the first target image reaches a preset standard; the image quality includes clarity and brightness; The second focus acquisition module is configured to, if not, acquire a point within a preset range centered on the first focus from the to-be-shot image based on the comparison result of different quality parameters of the first target image and a preset standard as a second focus, the first focus and the second focus are located on the subject; A refocusing module, configured to obtain the depth data corresponding to the second focus from the depth data of the to-be-shot image; take the depth data corresponding to the second focus as the distance from the shooting subject to the TOF camera ; Control the RGB camera to focus according to the distance from the shooting subject to the TOF camera; The re-shooting module is used for capturing the image to be captured by the focused RGB camera to obtain a second target image. 8. The device of claim 7, wherein The distance acquisition module is further configured to acquire the first focus from the to-be-shot image; acquiring depth data corresponding to the first focus from the depth data of the to-be-shot image; The depth data corresponding to the first focus is taken as the distance from the shooting subject to the TOF camera. 9. The device of claim 8, wherein The distance acquisition module is further configured to acquire a subject from the to-be-shot image; Determine a plurality of to-be-selected focal points from the photographic subject, and the plurality of to-be-selected focal points have different depth data; Shooting and previewing according to each of the to-be-selected focal points, respectively, to obtain a plurality of preview images; A target preview image is selected from the plurality of preview images, a focus to be selected corresponding to the target preview image is acquired, and the focus to be selected is used as the first focus. 10. The device of claim 7, wherein: The focusing module is also used to calculate the motor parameters of the RGB camera according to the distance from the shooting subject to the TOF camera; control the motor movement of the RGB camera according to the calculated motor parameters of the RGB camera to realize focusing . 11. The device of claim 7, wherein: The depth data acquisition module of the image to be shot is also used to obtain multiple frames of TOF data of different phases of the image to be shot through the TOF camera, and each phase corresponds to a frame of TOF data; The multiple frames of TOF data are synthesized to obtain the depth data of the image to be captured. 12. The apparatus of claim 7, wherein The depth data acquisition module of the to-be-captured image is further configured to perform filtering and denoising operations on the depth data of the to-be-captured image. 13. A computer-readable storage medium on which a computer program is stored, wherein the computer program realizes the steps of the method for controlling focusing according to any one of claims 1 to 6 when the computer program is executed by the processor . 14. An electronic device comprising a memory, a processor and a computer program stored on the memory and running on the processor, wherein the processor implements any one of claims 1 to 6 when the processor executes the computer program. The steps of a method of controlling focus.

Images