CN108377327B - Panoramic camera and depth information acquisition method - Google Patents

Abstract

The invention provides a panoramic camera and a method for acquiring depth information; the panoramic camera includes a plurality of combined camera mechanisms (1); the combined camera mechanism includes a first camera (11), a second camera (12), a first camera A processor (13) and a shutter release module (14); further comprising a light intensity adjustment module (16) for providing illumination to the second camera; the first processor is further configured to control focal length parameters of the first camera and the second camera , aperture parameters and shutter parameters, and obtain the first image captured by the first camera and multiple second images captured by the second camera under various light intensities through the triggering of the shutter release module; also used for According to the correspondence between the light intensity provided by the light intensity adjustment module and the depth of the image captured by the second camera under the light intensity, the depth information of each image area in the first image is acquired. The panoramic camera and the depth information acquisition method of the present invention have ingenious technical solutions and strong practicability.

Description

Panoramic camera and depth information acquisition method

technical field

The invention relates to the field of imaging equipment, in particular to a panoramic camera and a method for acquiring depth information.

Background technique

With the development of virtual reality technology, higher requirements are put forward for the interaction between people and virtual reality scenes. At present, images of virtual reality scenes based on real scenes are captured by panoramic cameras, such as google maps. However, the panoramic images shot by the existing panoramic cameras do not have depth information, and the three-dimensional sense of spatial depth is poor, which can only meet the requirements for panoramic shooting.

SUMMARY OF THE INVENTION

The present invention proposes a panoramic camera and a depth information acquisition method in order to solve the problem that the panoramic image captured by the existing panoramic camera has no depth information, has poor spatial depth and stereoscopic effect, and can only meet the requirements for panoramic photography.

The technical scheme proposed by the present invention is as follows:

The present invention provides a panoramic camera, which includes a plurality of combined camera mechanisms; the plurality of combined camera mechanisms are distributed on a circumference with a point as the center, and the plurality of combined camera mechanisms are mounted on a fixed bracket ; The combined camera mechanism includes a first camera, a second camera, a first processor and a shutter release module; the areas captured by the first camera and the second camera are the same; the combined camera mechanism also includes a circuit with the first processor a light intensity adjustment module for providing additional illumination to the second camera; the first processor is also electrically connected to the first camera, the second camera and the shutter release module, respectively, for controlling the first camera and the second camera focal length parameters, aperture parameters and shutter parameters, and through the triggering of the shutter release module, the first image captured by the first camera and the images captured by the second camera under various light intensities provided by the light intensity adjustment module are obtained. multiple second images; also used to compare the first image and multiple second images, and adjust the corresponding relationship between the light intensity provided by the light intensity module and the depth of the image captured by the second camera under the light intensity , so as to obtain the depth information of each image area in the first image.

In the above-mentioned panoramic camera of the present invention, the shutter release module includes a first shutter release for triggering the first camera to acquire the first image, and a second shutter release for triggering the second camera to acquire the second image; the first shutter release and the second shutter cable are respectively electrically connected with the first processor.

In the above-mentioned panoramic camera of the present invention, the plurality of second images captured by the second camera under N light intensities provided by the light intensity adjustment module from small to large are correspondingly denoted as F1, F2, F3, . . . , F (n-1), Fn, ..., F(N-1), FN; n<N; the first image is denoted as F0;

The first processor is used to compare F0 and F1, and determine the image area that can be covered by F1 in the first image; according to the difference between the light intensity provided by the light intensity adjustment module and the depth of the image captured by the second camera under the light intensity. The corresponding relationship determines the image depth corresponding to the illumination corresponding to F1; the image depth corresponding to the illumination corresponding to F1 is determined as the depth information of the image area that can be covered by F1 in the first image;

It is used to compare F1 and F2 to determine the image area in the first image that can be covered by F2 but not covered by F1; according to the difference between the light intensity provided by the light intensity adjustment module and the depth of the image captured by the second camera under the light intensity The corresponding relationship between determines the image depth corresponding to the illumination corresponding to F2; the image depth corresponding to the illumination corresponding to F2 is determined as the depth information of the image area that can be covered by F2 but not covered by F1 in the first image;

Used to compare F(n-1) and Fn to determine the image area in the first image that can be covered by Fn but not covered by F(n-1); adjust the light intensity provided by the module according to the light intensity and the second camera The correspondence between the image depths captured under this light intensity determines the image depth corresponding to the illumination corresponding to Fn; the image depth corresponding to the illumination corresponding to Fn is determined as the image depth in the first image that can be covered by Fn but not covered by Fn (n-1) depth information of the covered image area;

Used to compare F(N-1) and FN, and determine the image area in the first image that can be covered by FN but not covered by F(N-1); adjust the light intensity provided by the module according to the light intensity and the second camera. The corresponding relationship between the image depths captured under this light intensity determines the image depth corresponding to the illumination corresponding to FN; the image depth corresponding to the illumination corresponding to FN is determined as the image depth in the first image that can be covered by FN but not covered by F (N-1) Depth information of the covered image area.

In the above-mentioned panoramic camera of the present invention, the light intensity adjustment module is used for providing various light intensities in multiple consecutive time periods, and the second camera is used for shooting multiple second images respectively in the multiple time periods.

In the above-mentioned panoramic camera of the present invention, a second processor is further included, and the second processor is electrically connected to the first processors of the multiple combined camera mechanisms respectively, and is used for processing the first processors of the multiple combined camera mechanisms. A plurality of first images respectively captured by a camera are spliced to form a panoramic image.

In the above-mentioned panoramic camera of the present invention, the light intensity adjustment module is a flash or an external light source.

The present invention also proposes a method for acquiring depth information based on the above-mentioned panoramic camera, which is characterized by comprising the following steps:

Step S1, obtaining in advance the correspondence between the light intensity provided by the light intensity adjustment module and the depth of the image captured by the second camera under the light intensity;

Step S2, using the first camera to shoot the first image; and using the second camera to shoot multiple second images under the multiple light intensities provided by the light intensity adjustment module;

Step S3, compare the first image and a plurality of second images, and obtain the first image according to the corresponding relationship between the light intensity provided by the light intensity adjustment module and the depth of the image captured by the second camera under the light intensity. depth information of each image area.

In the above-mentioned depth information acquisition method of the present invention, the plurality of second images captured by the second camera under N light intensities from small to large provided by the light intensity adjustment module are correspondingly denoted as F1, F2, F3, . . . , F(n-1), Fn, ..., F(N-1), FN; the first image is marked as F0; step S3 also includes:

Step S31, compare F0 and F1, and determine the image area that can be covered by F1 in the first image; according to the corresponding relationship between the light intensity provided by the light intensity adjustment module and the depth of the image captured by the second camera under the light intensity Determine the image depth corresponding to the illumination corresponding to F1; determine the image depth corresponding to the illumination corresponding to F1 as the depth information of the image area that can be covered by F1 in the first image;

Step S32, compare F1 and F2, and determine the image area in the first image that can be covered by F2 but not covered by F1; according to the light intensity provided by the light intensity adjustment module and the depth of the image captured by the second camera under the light intensity The correspondence between determines the image depth corresponding to the illumination corresponding to F2; the image depth corresponding to the illumination corresponding to F2 is determined as the depth information of the image area that can be covered by F2 but not covered by F1 in the first image;

Step S33, compare F(n-1) and Fn, and determine the image area in the first image that can be covered by Fn but not covered by F(n-1); The corresponding relationship between the image depths captured by the camera under the light intensity determines the image depth corresponding to the illumination corresponding to Fn; the image depth corresponding to the illumination corresponding to Fn is determined as the first image that can be covered by Fn but not covered by Fn. The depth information of the image area covered by F(n-1);

Step S34, compare F(N-1) and FN, and determine the image area in the first image that can be covered by FN but not covered by F(N-1); The correspondence between the image depths captured by the camera under this light intensity determines the image depth corresponding to the illumination corresponding to the FN; the image depth corresponding to the illumination corresponding to the FN is determined as the image depth in the first image that can be covered by the FN but not covered by the FN. Depth information of the image area covered by F(N-1).

In the above-mentioned depth information acquisition method of the present invention, in step S2, the light intensity adjustment module is used to provide a variety of light intensities in multiple consecutive time periods, and the second camera is used to shoot multiple light intensities in the multiple time periods respectively. second image.

The panoramic camera of the present invention determines the depth information of the image area in the image by using the proportional relationship between the light intensity when the image is captured and the area of the effective area of the image. The panoramic camera and the depth information acquisition method of the present invention have ingenious technical solutions and strong practicability.

Description of drawings

The present invention will be further described below in conjunction with the accompanying drawings and embodiments, in which:

FIG. 1 shows a schematic diagram of a panoramic camera according to an embodiment of the present invention;

FIG. 2 shows a schematic diagram of functional modules of the panoramic camera shown in FIG. 1 .

Detailed ways

The technical problem to be solved by the present invention is: the panoramic image shot by the existing panoramic camera has no depth information, and the spatial depth stereoscopic effect is poor, which can only meet the requirements for panoramic shooting. The technical idea proposed by the present invention for this technical problem is: by using the proportional relationship between the light intensity when the image is captured and the area of the effective area of the image, the depth information of the image area in the image is determined.

In order to make the technical purpose, technical solutions and technical effects of the present invention clearer and facilitate those skilled in the art to understand and implement the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

As shown in FIG. 1 and FIG. 2 , FIG. 1 shows a schematic diagram of a panoramic camera according to an embodiment of the present invention. FIG. 2 shows a schematic diagram of functional modules of the panoramic camera shown in FIG. 1 . The panoramic camera includes multiple combined camera mechanisms 1; the three combined camera mechanisms 1 shown in FIG. 1 may be part or all of the multiple combined camera mechanisms 1; the multiple combined camera mechanisms 1 are in Distributed on a circle with one point as the center, and the multiple combined camera mechanisms 1 are all mounted on a fixed bracket (not shown in the figure); each combined camera mechanism 1 is responsible for shooting one in the 360° field of view Angle (the angle depends on the orientation of the combined camera mechanism 1).

Further, as shown in FIG. 2 , each combined camera mechanism includes a first camera 11 and a second camera 12 , and the first camera 11 and the second camera 12 have the same specifications. The areas captured by the first camera 11 and the second camera 12 are the same; the first camera 11 and the second camera 12 can be arranged left and right, up and down, or arranged in other positional relationships; as shown in FIG. 2, the combined type The camera mechanism further includes a first processor 13 and a shutter release module 14; the combined camera mechanism 1 further includes a light intensity adjustment module 16 that is electrically connected to the first processor 13 and used to provide additional illumination to the second camera 12; A processor 13 is also electrically connected to the first camera 11, the second camera 12 and the shutter release module 14, respectively, for controlling the focal length parameters, aperture parameters and shutter parameters of the first camera 11 and the second camera 12, and via the shutter The line module 14 is triggered to obtain the first image captured by the first camera 11 and multiple second images captured by the second camera 12 under various light intensities provided by the light intensity adjustment module 16; also use In order to compare the first image and a plurality of second images, and according to the corresponding relationship between the light intensity provided by the light intensity adjustment module 16 and the depth of the image captured by the second camera 12 under the light intensity, the first image is obtained. depth information of each image area. Here, according to the depth information of each image area in the first image, the three-dimensional reconstruction of the virtual reality scene can be realized by using the existing depth image registration method. In addition, in the present embodiment, the light intensity adjustment module 16 is a flash, and the light intensity used by the second camera 12 is realized by compensating the illumination with the flash. It can be understood that, in other embodiments, the light intensity used by the second camera 12 may also be realized by an external light source. In this embodiment, the shutter release module 14 includes a first shutter release 141 for triggering the first camera 11 to acquire a first image, and a second shutter release 142 for triggering the second camera 12 to acquire a second image; A shutter release 141 and a second shutter release 142 are electrically connected to the first processor 13 respectively.

The present invention also proposes a method for acquiring depth information of an object based on the above-mentioned panoramic camera, comprising the following steps:

Step S1, obtaining in advance the correspondence between the light intensity provided by the light intensity adjustment module 16 and the depth of the image captured by the second camera 12 under the light intensity;

Step S2, using the first camera 11 to shoot the first image; and using the second camera 12 to shoot multiple second images under the various light intensities provided by the light intensity adjustment module 16; in order to save time, the light intensity adjustment module can be used 16. Provide various light intensities in multiple consecutive time periods, and then use the second camera (12) to separately capture second images in the multiple time periods to complete the shooting task. Meanwhile, in order to more accurately determine the depth information of the image area, the light intensity can be more precisely defined.

Step S3, compare the first image and a plurality of second images, and obtain the first image according to the corresponding relationship between the light intensity provided by the light intensity adjustment module 16 and the depth of the image captured by the second camera 12 under the light intensity. Depth information for each image region in the image. In this step, the light intensity is proportional to the effective area of the effective image captured by the second camera 12, that is, the stronger the light intensity, the greater the depth of the image captured by the second camera 12 at the light intensity.

In order to explain step S3 more clearly, in this embodiment, the plurality of second images captured by the second camera 12 under N light intensities from small to large provided by the light intensity adjustment module 16 correspond to Denoted as F1, F2, F3, ..., F(n-1), Fn, ..., F(N-1), FN; denote the first image as F0; step S3 further includes:

Step S31, compare F0 and F1, and determine the image area that can be covered by F1 in the first image; according to the light intensity provided by the light intensity adjustment module 16 and the depth of the image captured by the second camera 12 under the light intensity. The corresponding relationship determines the image depth corresponding to the illumination corresponding to F1; the image depth corresponding to the illumination corresponding to F1 is determined as the depth information of the image area in the first image that can be covered by F1; here, the meaning of coverage refers to the existence of Intersection; the image area covered by the second image in the first image refers to the image area in the first image that corresponds to the image area where the definition of the second image reaches the level of the first image. The evaluation of the sharpness of the image can be carried out through spatial parameter variance, entropy, and frequency domain modulation transfer function MTF, etc., or can be done manually by employing multiple evaluators.

Step S32, compare F1 and F2, and determine the image area that can be covered by F2 but not covered by F1 in the first image; according to the light intensity provided by the light intensity adjustment module 16 and the light intensity captured by the second camera 12 under the light intensity The correspondence between the image depths determines the image depth corresponding to the illumination corresponding to F2; the image depth corresponding to the illumination corresponding to F2 is determined as the depth information of the image area in the first image that can be covered by F2 but not covered by F1 ;

Step S33, compare F(n-1) and Fn, and determine the image area in the first image that can be covered by Fn but not covered by F(n-1); The corresponding relationship between the image depths captured by the two cameras 12 under the light intensity determines the image depth corresponding to the illumination corresponding to Fn; the image depth corresponding to the illumination corresponding to Fn is determined as the first image that can be covered by Fn but Depth information of the image area not covered by F(n-1);

Step S34, compare F(N-1) and FN, and determine the image area in the first image that can be covered by FN but not covered by F(N-1); The correspondence between the image depths captured by the two cameras 12 under the light intensity determines the image depth corresponding to the illumination corresponding to the FN; Depth information for image regions not covered by F(N-1).

In this way, steps S31 to S34 are executed and completed by the first processor 13; through steps S31 to S34, the depth information of each image area in the first image can be obtained. Here, from the depth information of each image area, the depth information of the object, space or person occupying the image area can be obtained.

Further, the panoramic camera further includes a second processor 15, the second processor 15 is electrically connected to the first processors 13 of the multiple combined camera mechanisms 1 respectively, and is used for processing the multiple combined camera mechanisms 1. The plurality of first images respectively captured by the first cameras 11 are stitched together to form a panoramic image.

Through the panoramic camera and the depth information acquisition method, a preliminary model of virtual reality can be established, thereby providing a more realistic three-dimensional sense and spatial distance sense for the virtual reality scene. The technical solution of the present invention can be applied to various applications of virtual reality based on the 3D modeling information generated in the real environment; compared with the traditional stereoscopic image information, the 3D modeling information with depth information provided by this proposal can be more The user provides a real sense of spatial distance. Furthermore, it can be combined with the interactive technology of virtual reality to realize the interactive operation between people and the environment.

The panoramic camera of the present invention determines the depth information of the image area in the image by using the proportional relationship between the light intensity when the image is captured and the area of the effective area of the image. The panoramic camera and the depth information acquisition method of the present invention have ingenious technical solutions and strong practicability.

It should be understood that, for those skilled in the art, improvements or changes can be made according to the above description, and all these improvements and changes should fall within the protection scope of the appended claims of the present invention.

Claims (7)

1. A panoramic camera, characterized in that it comprises a plurality of combined camera mechanisms (1); the plurality of combined camera mechanisms (1) are distributed and arranged on a circumference centered on one point, and the plurality of combined camera mechanisms The mechanisms (1) are all mounted on a fixed bracket; the combined camera mechanism (1) includes a first camera (11), a second camera (12), a first processor (13) and a shutter release module (14) The first camera (11) and the second camera (12) take the same area; the combined camera mechanism (1) also includes an electrical connection with the first processor (13) for feeding the second camera (12) A light intensity adjustment module (16) for providing additional illumination; the first processor (13) is further electrically connected to the first camera (11), the second camera (12) and the shutter release module (14) respectively, for controlling the first camera (11), the second camera (12) and the shutter release module (14) respectively The focal length parameters, aperture parameters and shutter parameters of a camera (11) and a second camera (12), and the first image F0 captured by the first camera (11) and the first image F0 captured by the first camera (11) and the Multiple second images captured by the second camera (12) under multiple light intensities provided by the light intensity adjustment module (16); also used for comparing the first image and the multiple second images, and according to the light intensity Adjust the multiple second images under N light intensities from small to large provided by the module (16), and denote the multiple second images as F1, F2, F3, . . . , F(n-1), Fn , . The corresponding relationship between the light intensity and the depth of the image captured by the second camera (12) under the light intensity determines the image depth corresponding to the illumination corresponding to F1; the image depth corresponding to the illumination corresponding to F1 is determined as the first image The depth information of the image area that can be covered by F1; For comparing F1 and F2, to determine the image area in the first image that can be covered by F2 but not covered by F1; according to the light intensity provided by the light intensity adjustment module (16) and the second camera (12) at the light intensity The corresponding relationship between the image depths captured in the next step determines the image depth corresponding to the illumination corresponding to F2; the image depth corresponding to the illumination corresponding to F2 is determined as the image area in the first image that can be covered by F2 but not covered by F1 depth information; Used to compare F(n-1) and Fn, to determine the image area in the first image that can be covered by Fn but not covered by F(n-1); according to the light intensity provided by the light intensity adjustment module (16) and The corresponding relationship between the image depths captured by the second camera (12) under the light intensity determines the image depth corresponding to the illumination corresponding to Fn; the image depth corresponding to the illumination corresponding to Fn is determined as a The depth information of the image area covered by Fn but not covered by F(n-1); Used to compare F(N-1) and FN, and determine the image area in the first image that can be covered by FN but not covered by F(N-1); according to the light intensity provided by the light intensity adjustment module (16) and The correspondence between the image depths captured by the second camera (12) under the light intensity determines the image depth corresponding to the illumination corresponding to the FN; the image depth corresponding to the illumination corresponding to the FN is determined as a Depth information for image regions covered by FN but not covered by F(N-1). 2. The panoramic camera according to claim 1, wherein the shutter release module (14) comprises a first shutter release (141) for triggering the first camera (11) to acquire the first image, and a first shutter release (141) for triggering the first camera (11) to acquire the first image, and for triggering the first shutter release (141) The two cameras (12) acquire a second shutter release (142) of the second image; the first shutter release (141) and the second shutter release (142) are respectively electrically connected to the first processor (13). 3. The panoramic camera according to claim 2, characterized in that, the light intensity adjustment module (16) is used to provide a variety of light intensities in a plurality of consecutive time periods, and the second camera (12) is used to A plurality of second images are respectively captured in the time period. 4 . The panoramic camera according to claim 1 , further comprising a second processor ( 15 ), the second processor ( 15 ) being respectively connected with the first processors of the plurality of combined camera mechanisms ( 1 ). 5 . The electrical connection of (13) is used for splicing a plurality of first images respectively captured by the first cameras (11) of the plurality of combined camera mechanisms (1) to form a panoramic image. 5. The panoramic camera according to claim 1, wherein the light intensity adjustment module (16) is a flash or an external light source. 6. A method for acquiring depth information based on a panoramic camera according to claim 1, characterized in that, comprising the following steps: Step S1, pre-acquire the corresponding relationship between the light intensity provided by the light intensity adjustment module (16) and the depth of the image captured by the second camera (12) under the light intensity; Step S2, using the first camera (11) to shoot the first image F0; and using the second camera (12) to shoot a plurality of second images under N light intensities from small to large provided by the light intensity adjustment module (16) Images and labeled as F1, F2, F3, ..., F(n-1), Fn, ..., F(N-1), FN; n<N; Step S3, compare and compare F0 and F1, and determine the image area that can be covered by F1 in the first image; according to the light intensity provided by the light intensity adjustment module (16) and the light intensity captured by the second camera (12) under the light intensity The correspondence between the image depths determines the image depth corresponding to the illumination corresponding to F1; the image depth corresponding to the illumination corresponding to F1 is determined as the depth information of the image area that can be covered by F1 in the first image; Step S4, compare F1 and F2, and determine the image area in the first image that can be covered by F2 but not covered by F1; The corresponding relationship between the image depths captured under strong conditions determines the image depth corresponding to the illumination corresponding to F2; the image depth corresponding to the illumination corresponding to F2 is determined as the image in the first image that can be covered by F2 but not covered by F1 Depth information of the region; Step S5, compare F(n-1) and Fn, and determine the image area in the first image that can be covered by Fn but not covered by F(n-1); according to the light intensity provided by the light intensity adjustment module (16) The corresponding relationship with the depth of the image captured by the second camera (12) under the light intensity determines the depth of the image corresponding to the illumination corresponding to Fn; The depth information of the image area covered by Fn but not covered by F(n-1); Step S6, compare F(N-1) and FN, and determine the image area in the first image that can be covered by FN but not covered by F(N-1); according to the light intensity provided by the light intensity adjustment module (16) The corresponding relationship with the depth of the image captured by the second camera (12) under the light intensity determines the depth of the image corresponding to the illumination corresponding to the FN; the depth of the image corresponding to the illumination corresponding to the FN is determined as a Depth information for image regions covered by FN but not covered by F(N-1). 7. The depth information acquisition method according to claim 6, characterized in that, in step S2, the light intensity adjustment module (16) is used to provide multiple light intensities in a plurality of continuous time periods, and the second camera (12 ) is used to capture a plurality of second images respectively in the plurality of time periods.

Images