CN107025666A - Single-camera-based depth detection method, device, and electronic device - Google Patents

Abstract

The embodiment of the invention discloses a depth detection method based on a single camera. The depth detection method comprises the following steps: controlling a camera to shoot a current scene at a first position to obtain a first image; controlling the camera to move from the first position to the second position along the direction perpendicular to the axial direction of the camera, and then controlling the camera to shoot a second image of the current scene; the first image and the second image are processed to obtain depth information of the current scene. The invention also discloses a depth measuring device and an electronic device of the single camera. According to the depth detection method of the single camera, the depth detection device of the single camera and the electronic device, disclosed by the embodiment of the invention, the image shooting is carried out on the basis of the movable single camera, so that on one hand, a relatively matched stereo image pair can be obtained, on the other hand, the limitation on the relative position of the camera during the stereo image pair shooting is small, the moving freedom degree of the camera is large, and the problem of inaccurate depth detection caused by the change of the relative position can be avoided.

Description

Single-camera-based depth detection method, device, and electronic device

technical field

The present invention relates to imaging technology, in particular to a depth detection method and device based on a single camera and an electronic device.

Background technique

In the depth detection method based on binocular stereo vision, it is necessary to ensure the specifications of the two cameras to obtain a relatively matching stereo image pair, and to ensure that the relative positions between the two cameras are fixed, so as to ensure the accuracy of the depth data. However, it is difficult to ensure that the specifications of the two cameras are exactly the same in the current production of the camera. In addition, the relative position of the two cameras may change due to reasons such as falling, so that more accurate depth data cannot be obtained.

Contents of the invention

Embodiments of the present invention provide a method and device for depth detection based on a single camera, and an electronic device.

A depth detection method for a single camera according to an embodiment of the present invention. The depth detection method comprises the following steps:

controlling the camera to capture the current scene at a first position to obtain a first image;

controlling the camera to move from the first position to a second position in a direction perpendicular to the axial direction of the camera; and then controlling the camera to capture a second image of the current scene; and

Processing the first image and the second image to obtain depth information of the current scene.

The depth detection device of a single camera according to the embodiment of the present invention is used in an electronic device. The depth detection device includes a first control module, a second control module and a processing module. The first control module is used to control the camera to capture the current scene at a first position to obtain a first image; the second control module is used to control the camera from the first position along a direction perpendicular to the camera After the axial direction is moved to the second position, the camera is controlled to take a second image of the current scene; the processing module is used to process the first image and the second image to obtain the depth of the current scene information.

An electronic device according to an embodiment of the present invention includes a camera, a motion sensor, and the above-mentioned depth detection device. The depth detection device is electrically connected to both the camera and the motion sensor.

The single-camera-based depth detection method, the single-camera-based depth detection device, and the electronic device of the embodiments of the present invention perform image capture based on a movable single camera. On the one hand, relatively matching stereoscopic image pairs can be obtained; There are less limitations on the relative position of the camera during shooting, and the freedom of movement of the camera is large, which can avoid the problem of inaccurate depth detection caused by relative position changes.

Additional aspects and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.

Description of drawings

The above and/or additional aspects and advantages of the present invention will become apparent and comprehensible from the description of the embodiments in conjunction with the following drawings, wherein:

Fig. 1 is a schematic flow chart of a depth detection method based on a single camera according to an embodiment of the present invention;

2 is a schematic diagram of functional modules of an electronic device according to an embodiment of the present invention;

3 is a schematic diagram of the state of the depth detection method in some embodiments of the present invention;

4 is a schematic diagram of the state of the depth detection method in some embodiments of the present invention;

5 is a schematic diagram of the state of the depth detection method in some embodiments of the present invention;

Fig. 6 is a schematic flowchart of a depth detection method in some embodiments of the present invention;

7 is a schematic diagram of functional modules of a second control module in some embodiments of the present invention;

Fig. 8 is a schematic flowchart of a depth detection method in some embodiments of the present invention;

Fig. 9 is a schematic diagram of functional modules of a second control module in some embodiments of the present invention;

10 is a schematic diagram of the state of the depth detection method in some embodiments of the present invention;

Fig. 11 is a schematic flowchart of a depth detection method in some embodiments of the present invention;

Figure 12 is a functional block diagram of a processing module in some embodiments of the present invention; and

Fig. 13 is a schematic diagram of the state of the depth detection method in some embodiments of the present invention.

detailed description

Embodiments of the present invention are described in detail below, examples of which are shown in the drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary only for explaining the present invention and should not be construed as limiting the present invention.

Please refer to FIGS. 1 to 2 together. The depth detection method based on a single camera in the embodiment of the present invention includes the following steps:

S11: Control the camera 20 to capture the current scene at the first position to obtain the first image;

S12: Control the camera 20 to move from the first position to the second position in a direction perpendicular to the axial direction of the camera 20, and then control the camera 20 to capture a second image of the current scene;

S13: Process the first image and the second image to obtain depth information of the current scene.

The single-camera-based depth detection method in the embodiment of the present invention is used for a single-camera depth detection device 10 . The depth detection device 10 based on a single camera includes a first control module 11 , a second control module 12 and a processing module 13 . Step S11 can be realized by the first control module 11 , step S12 can be realized by the second control module 12 , and step S13 can be realized by the processing module 13 .

That is to say, the first control module 11 is used to control the camera 20 to capture the current scene at the first position to obtain the first image; the second control module 12 is used to control the camera 20 from the first position along the axis perpendicular to the camera After moving to the second position in the direction of the control camera 20 to take a second image of the current scene; the processing module 13 is used to process the first image and the second image to obtain the depth information of the current scene.

The single-camera-based depth detection device 10 in the embodiment of the present invention is applied to the electronic device 100 in the embodiment of the present invention. That is to say, the electronic device 100 in the embodiment of the present invention includes the depth detection device 10 based on a single camera in the embodiment of the present invention. Of course, the electronic device 100 in the embodiment of the present invention also includes a camera 20 and a motion sensor 30 . Wherein, the depth detection device 10 is electrically connected to the camera 20 and the motion sensor 30 .

In some embodiments, the electronic device 100 includes a mobile phone, a tablet computer, a laptop computer, a smart watch, a smart bracelet, smart glasses, etc., without any limitation here. In a specific embodiment of the present invention, the electronic device 100 is a mobile phone.

Specifically, please refer to Figures 3 to 5, where X ^l is the shortest distance from the midpoint of the bottom of the cup in the image to the leftmost end of the image, X ^r is the shortest distance from the midpoint of the bottom of the cup in the image to the rightmost end of the image, and C is the distance of the image width. Specifically, the first image is captured at the first position O1, and then the camera 20 is moved to the second position Or to capture the second image. Wherein, the direction of the connecting line between the first position O1 and the second position Or is perpendicular to the axial direction of the camera 20 . In this way, a stereoscopic image pair of the first image and the second image is obtained, and then the stereoscopic image pair is processed to obtain the depth information of the current scene.

It should be noted that the axial direction of the camera 20 refers to a direction parallel to the direction of the optical axis when the camera 20 captures the current scene.

It can be understood that the depth detection method based on binocular stereo vision is difficult to ensure that the specifications of the two cameras are completely consistent, and the relative position between the two cameras may change due to external factors such as the fall of the electronic device 100, thereby affecting the depth detection. accuracy. The depth detection method for a single camera in the embodiment of the present invention is based on a movable single camera for image capture. On the one hand, a relatively matching stereo image pair can be obtained; , the camera has a large degree of freedom of movement, which can avoid the problem of inaccurate depth detection caused by relative position changes.

Referring to FIG. 6, in some implementations, step S12 controls the camera 20 to move from the first position to the second position along the direction perpendicular to the axial direction of the camera 20, and then controls the camera 20 to capture the second image of the current scene. The following steps:

S121: Determine whether the camera 20 moves from the first position to a second position in a direction perpendicular to the axial direction of the camera 20 according to the detection data of the motion sensor 30;

S122: Control the camera 20 to capture a second image when the camera 20 moves from the first position to a second position in a direction perpendicular to the axial direction of the camera 20; and

S123: Control the electronic device 100 to issue a prompt when the camera 20 does not move from the first position to the second position along the direction perpendicular to the axial direction of the camera 20 .

Referring to FIG. 7 , in some implementations, the second control module 12 includes a judgment unit 121 , a control unit 122 and a prompt unit 123 . Step S121 can be implemented by the judging unit 121 , step S122 can be implemented by the control unit 122 , and step S123 can be implemented by the prompting unit 123 .

That is to say, the judging unit 121 is used to determine whether the camera 20 moves from the first position to the second position along the direction perpendicular to the axial direction of the camera 20 according to the detection data of the motion sensor 30; When moving from the first position along the direction perpendicular to the axial direction of the camera 20 to the second position, the camera 20 is controlled to take a second image; When the direction moves to the second position, the control electronic device 100 sends out a prompt.

Please refer to 8. Specifically, a space Cartesian coordinate system X-Y-Z is established in the space where the electronic device 100 is located. In a specific embodiment of the present invention, the axial direction of the camera 20 can be regarded as the Y-axis direction in the space Cartesian coordinate system. The X-axis direction is the first moving direction perpendicular to the axial direction of the camera 20, the Y-axis is the second moving direction parallel to the axial direction of the camera 20, and the Z-axis is the third moving direction perpendicular to the axial direction of the camera 20. direction of movement. In order to keep the camera 20 moving from the first position 01 in a direction perpendicular to the axial direction of the camera 20, the camera 20 is expressed in the X-axis direction, that is, the first moving direction, or in the Z-axis direction, that is, the second moving direction in the space Cartesian coordinate system. Move in three moving directions. In this way, two frames of images taken at different positions are obtained to obtain a stereoscopic image pair, and the depth information can be obtained by processing the stereoscopic image pair in a subsequent step. If the moving direction of the camera deviates from the X-axis direction or the Z-axis direction, the electronic device 100 needs to send a prompt message to remind the user to find the correct shooting position so as to ensure obtaining a qualified stereoscopic image pair.

In some embodiments, motion sensor 30 includes a gyroscope or an acceleration sensor. In a particular embodiment of the invention, the motion sensor 30 is a gyroscope.

It can be understood that the gyroscope can detect the deflection state of the electronic device 100 . The detection data of the gyroscope assists the user in correcting the moving direction of the camera 20 and determining the second position Or, so as to obtain a second image that can be used for acquiring depth information in a subsequent step.

Referring to FIG. 9, in some implementations, step S122 controls the camera 20 to capture a second image when the camera 20 moves from the first position to the second position along the direction perpendicular to the axial direction of the camera 20, including the following steps:

S1221: Detect whether the current scene is in a moving state when the camera 20 moves from the first position to a second position in a direction perpendicular to the axis of the camera 20; and

S1222: Control the camera 20 to capture a second image of the current scene when the current scene is not in motion.

Referring to FIG. 10 , in some implementations, the control unit 122 includes a detection subunit 1221 and a control subunit 1222 . Step S1221 can be implemented by the detection subunit 1221 , and step S1222 can be implemented by the control subunit 1222 .

That is to say, the detection subunit 1221 is used to detect whether the current scene is in a moving state when the camera 20 moves from the first position to the second position along the direction perpendicular to the axial direction of the camera 20; When the scene is not in motion, the camera 20 is controlled to capture the second image of the current scene.

It can be understood that when the camera 20 moves to the second position along the direction perpendicular to the axial direction of the camera 20, if the current scene is in a moving state, that is, people or objects in the current scene are moving, then the first shot taken by the camera 20 Second, the image may be blurry. In this way, the matching pixels corresponding to the feature points in the blurred second image may not be identified in subsequent steps, so that the first image and the second image cannot be used to obtain depth information. Therefore, when the current scene is in a moving state, the shooting process is temporarily not performed. If the current scene is not in motion, the camera 20 is controlled to capture the current scene to obtain the second image.

Please refer to FIG. 11 , in some embodiments, the step S13 of processing the first image and the second image to obtain the depth information of the current scene includes the following steps:

S131: Obtain the linear distance between the first position and the second position according to the detection data of the motion sensor 30;

S132: Obtain the feature points of the focused subject of the current scene on the first image and the second image and matching pixels of the corresponding feature points;

S133: Calculate the depth information of the focused subject according to the parameters of the camera 20, the linear distance, the coordinates of the feature points and the matched pixels.

Referring to FIG. 12 , in some implementations, the processing module 13 includes a first acquisition unit 131 , a second acquisition unit 132 and a calculation unit 133 . Step S131 can be implemented by the first acquisition unit 131 , step S132 can be implemented by the second acquisition unit 132 , and step S133 can be implemented by the calculation unit 133 .

That is to say, the acquisition unit 131 is used to acquire the linear distance between the first position and the second position according to the detection data of the motion sensor 30; the second acquisition unit 132 is used to acquire the focus subject of the current scene on the first image and the matching pixels of the feature points and the corresponding feature points on the second image; the calculation unit 133 is used to calculate the focus subject according to the parameters of the camera, the linear distance and the coordinates of the feature points and the matching pixels depth information.

Please refer to FIG. 4 , FIG. 5 and FIG. 13 together. In a specific embodiment of the present invention, the moving direction of the camera 20 is translation along the X-axis direction. In the figure, P is a certain pixel point of the focus subject that the user pays attention to in the current scene, and F is the focal length of the camera 20 when shooting the first image and the second image. It can be understood that the depth information corresponding to the current scene is formed by combining the depth information of multiple pixels, that is to say, the depth information of the current scene needs to be obtained by obtaining the depth information of multiple pixels. In addition, when the camera 20 is at the first position O1 and the second position Or, the framing pictures of the camera 20 are basically the same, that is to say, when the camera 20 is respectively at the first position O1 and the second position Or, the focus subject of the current scene They are all located in the viewfinder screen of the camera 20. In this way, the feature point recognition and the matching of the pixel points corresponding to the feature points are performed on the captured first image and the second image, wherein the feature point is the focus subject that the user pays attention to in the current scene Feature points, for example, the midpoint of the bottom of the cup shown in Figure 4, which is the first image, and Figure 5, which is the second image, is the feature point to be recognized, and the pixel corresponding to the midpoint of the bottom of the cup in the two images is the matching pixel . Then, the depth information of each matching pixel is calculated according to the parameters of the camera 20 , the linear distance S, the identified feature points and the coordinates of the matching pixels. Specifically, the linear distance S between the first position O1 and the second position Or can be obtained according to the detection data of the motion sensor 30, and then the depth information D of a matching pixel corresponding to a certain feature point in the image can be calculated according to the linear distance S . The calculation method of depth information D is: In this way, after the depth information D of each matching pixel in the current scene is calculated by using the above calculation formula, the depth information corresponding to the current scene can be composed of these matching pixels with depth information.

In other embodiments, the camera 20 can also move from the first position to the second position along the Z axis. Subsequently, the depth information is still calculated according to information such as the linear distance S between the first position and the second position, parameters of the camera 20 , and coordinates of feature points and matching pixels.

The electronic device 100 also includes a housing, a memory, a circuit board, and a power supply circuit. Wherein, the circuit board is placed inside the space surrounded by the casing, and the processor and memory are arranged on the circuit board; the power supply circuit is used to supply power to each circuit or device of the electronic device 100; the memory is used to store executable program codes; the depth detection The device 10 executes the program corresponding to the executable program code by reading the executable program code stored in the memory, so as to implement the above-mentioned depth detection method in any embodiment of the present invention.

In the description of this specification, reference to the terms "one embodiment", "some embodiments", "exemplary embodiments", "example", "specific examples", or "some examples" etc. A specific feature, structure, material, or characteristic described in an embodiment or an example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the described specific features, structures, materials or characteristics may be combined in any suitable manner in any one or more embodiments or examples.

Any process or method descriptions in flowcharts or otherwise described herein may be understood to represent modules, segments or portions of code comprising one or more executable instructions for implementing specific logical functions or steps of the process , and the scope of preferred embodiments of the invention includes alternative implementations in which functions may be performed out of the order shown or discussed, including substantially concurrently or in reverse order depending on the functions involved, which shall It is understood by those skilled in the art to which the embodiments of the present invention pertain.

The logic and/or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced listing of executable instructions for implementing logical functions, can be embodied in any computer-readable medium, For use with instruction execution systems, devices, or devices (such as computer-based systems, systems including processors, or other systems that can fetch instructions from instruction execution systems, devices, or devices and execute instructions), or in conjunction with these instruction execution systems, devices or equipment used. For the purposes of this specification, a "computer-readable medium" may be any device that can contain, store, communicate, propagate or transmit a program for use in or in conjunction with an instruction execution system, device or device. More specific examples (non-exhaustive list) of computer-readable media include the following: electrical connection with one or more wires (electronic device), portable computer disk case (magnetic device), random access memory (RAM), Read Only Memory (ROM), Erasable and Editable Read Only Memory (EPROM or Flash Memory), Fiber Optic Devices, and Portable Compact Disc Read Only Memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program can be printed, since the program can be read, for example, by optically scanning the paper or other medium, followed by editing, interpretation or other suitable processing if necessary. The program is processed electronically and stored in computer memory.

It should be understood that various parts of the present invention can be realized by hardware, software, firmware or their combination. In the embodiments described above, various steps or methods may be implemented by software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one or combination of the following techniques known in the art: Discrete logic circuits, ASICs with suitable combinational logic gates, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

Those of ordinary skill in the art can understand that all or part of the steps carried by the methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. During execution, one or a combination of the steps of the method embodiments is included.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing module, each unit may exist separately physically, or two or more units may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software function modules. If the integrated modules are realized in the form of software function modules and sold or used as independent products, they can also be stored in a computer-readable storage medium.

The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, and the like. Although the embodiment of the present invention has been shown and described above, it can be understood that the above embodiment is exemplary and should not be construed as a limitation of the present invention, and those skilled in the art can make the above-mentioned The embodiments are subject to changes, modifications, substitutions and variations.

Claims (11)

1. A depth detection method based on a single camera, characterized in that, the depth detection method comprises the following steps: controlling the camera to capture the current scene at a first position to obtain a first image; controlling the camera to move from the first position to a second position in a direction perpendicular to the axial direction of the camera; and then controlling the camera to capture a second image of the current scene; and Processing the first image and the second image to obtain depth information of the current scene. 2. The depth detection method according to claim 1, wherein the camera is applied to an electronic device, the electronic device includes a motion sensor, and the control of the camera is from the first position along the vertical direction to the After the axial direction of the camera is moved to the second position, the step of controlling the camera to capture the second image of the current scene includes the following steps: determining whether the camera moves from the first position to the second position in a direction perpendicular to the axial direction of the camera according to the detection data of the motion sensor; controlling the camera to capture the second image when the camera moves from the first position to the second position in a direction perpendicular to the axial direction of the camera; and When the camera does not move from the first position to the second position in a direction perpendicular to the axial direction of the camera, the electronic device is controlled to issue a prompt. 3. The depth detection method according to claim 2, wherein when the camera moves from the first position to the second position in a direction perpendicular to the axial direction of the camera, the The step of taking the second image by the camera also includes the following steps: detecting whether the current scene is in a motion state when the camera moves from the first position to the second position in a direction perpendicular to the axial direction of the camera; and and controlling the camera to capture a second image of the current scene when the current scene is not in a moving state. 4. The depth detection method according to claim 2, wherein the step of processing the first image and the second image to obtain the depth information of the current scene comprises the following steps: acquiring a linear distance between the first position and the second position according to the detection data of the motion sensor; Acquiring feature points of the focused subject of the current scene on the first image and on the second image and matching pixels corresponding to the feature points; and Depth information of the focused subject is calculated according to the parameters of the camera, the linear distance, and coordinates of the feature points and the matching pixels. 5. A depth detection device based on a single camera for an electronic device, wherein the depth detection device comprises: A first control module, the first control module is used to control the camera to capture the current scene at the first position to obtain the first image; A second control module, the second control module is configured to control the camera to capture the current scene after the camera moves from the first position to a second position in a direction perpendicular to the axis of the camera the second image; and A processing module, configured to process the first image and the second image to obtain depth information of the current scene. 6. The depth detection device according to claim 5, wherein the camera is applied to an electronic device, the electronic device includes a motion sensor, and the second control module includes: A judging unit, configured to determine whether the camera moves from the first position to the second position in a direction perpendicular to the axial direction of the camera according to the detection data of the motion sensor; a control unit configured to control the camera to capture the second image when the camera moves from the first position to the second position in a direction perpendicular to the axial direction of the camera; and A prompting unit, configured to control the electronic device to issue a prompt when the camera does not move from the first position to the second position in a direction perpendicular to the axial direction of the camera. 7. The depth detection device according to claim 6, wherein the control unit comprises: A detection subunit, configured to detect whether the current scene is in a moving state when the camera moves from the first position to the second position in a direction perpendicular to the axial direction of the camera; with A control subunit, configured to control the camera to capture a second image of the current scene when the current scene is not in a moving state. 8. The depth detection device according to claim 6, wherein the processing module comprises: a first acquiring unit, configured to acquire the linear distance between the first position and the second position according to the detection data of the motion sensor; A second acquisition unit, configured to acquire feature points on the first image and on the second image of the focus subject of the current scene and matching pixels corresponding to the feature points; and A calculation unit, configured to calculate the depth information of the focused subject according to the parameters of the camera, the linear distance, and the coordinates of the feature points and the matching pixels. 9. The depth detection device according to claim 6, the motion sensor comprises a gyroscope or an acceleration sensor. 10. An electronic device, characterized in that the electronic device comprises: Camera; motion sensor; and The depth detection device according to any one of claims 5 to 9, wherein the depth detection device is electrically connected to both the camera and the motion sensor. 11. The electronic device according to claim 10, wherein the electronic device comprises a mobile phone and/or a tablet computer.