WO2018121401A1

WO2018121401A1 - Splicing method for panoramic video images, and panoramic camera

Info

Publication number: WO2018121401A1
Application number: PCT/CN2017/117711
Authority: WO
Inventors: 张燕; 胡治国; 李炜
Original assignee: 深圳市掌网科技股份有限公司
Priority date: 2016-12-28
Filing date: 2017-12-21
Publication date: 2018-07-05
Also published as: CN108616733A; CN108616733B

Abstract

The present invention relates to the technical field of virtual reality, and in particular relates to a splicing method for panoramic video images, and a panoramic camera. Provided is a splicing method for panoramic video images. The method comprises the following steps: S1, acquiring a plurality of panoramic video images; S2, acquiring depth information about an overlap area in each panoramic video image; S3, according to the variation degree of the depth information about the overlap area, dividing same into an area where the variation degree of the depth information is low and an area where the variation degree of the depth information is high; S4, performing image splicing once on the area of the overlap area where the variation degree of the depth information is low, and performing frame-by-frame image splicing on the area of the overlap area where the variation degree of the depth information is high; and S5, generating a spliced panoramic video image. The present invention can acquire depth information about a certain point or a certain area in a stereoscopic image rapidly and accurately, and can reduce the information data processing capacity and complexity in the panoramic video image splicing of a device, thereby improving the splicing efficiency of an overlap area, with the characteristics of fast and real-time splicing.

Description

Mosaic method of panoramic video image and panoramic camera

Technical field

The present invention relates to the field of virtual reality technologies, and in particular, to a method for stitching panoramic video images and a panoramic camera.

Background technique

The popularity of digital imaging devices has made digital images widely available. In actual scientific research and engineering projects, ultra-wide viewing angle images that exceed the human eye's angle of view are often used, but due to the limitation of the distance, the angle of view of ordinary digital imaging equipment is usually unable to capture large-sized objects with a single photo. . In order to obtain high-resolution images with large viewing angles, wide-angle lenses are generally used for shooting. However, in some special circumstances, the limitation of the device itself makes the image field width of the image not meet the application requirements. For example, a wide-angle lens can be used to obtain a wide-angle image. However, the edges of the wide-angle lens produce inevitable distortion, and the wide-angle lens is expensive and expensive to use.

With the development of computer and image processing technology, image stitching technology provides a good solution for obtaining wide viewing angle images. The image stitching technique can combine a series of small-view images with partially overlapping boundaries and use different processing algorithms to match and align according to the characteristics of different images, thereby splicing into a wide-angle image. However, the traditional panoramic image stitching method is complicated, the efficiency is not high, and the equipment is expensive and inconvenient to use, it is necessary to provide a convenient and quick and easy to use panoramic image stitching method and device.

technical problem

In order to solve the above technical problem, the present invention provides a splicing method for a panoramic video image and a panoramic camera, which only splicing a region with a large change in depth information of overlapping regions in each panoramic video image, thereby reducing the information processing amount of the device. Increase the stitching speed.

Technical solution

The technical solution adopted by the present invention is to provide a method for splicing a panoramic video image, which includes the following steps:

S1 acquires a plurality of panoramic video images, which are composed of a plurality of stereoscopic images captured in different directions of the same circumference, and an effective image capturing space of the adjacent stereoscopic images has an edge overlapping overlapping region;

S2 acquiring depth information of an overlapping area in each panoramic video image, where the depth information is used to indicate a spatial position of each pixel in the panoramic video image with respect to the shooting position;

S3 divides the depth information into a region where the depth information changes little and a region where the depth information changes greatly according to the degree of change of the depth information of the overlapping region; the depth information changes to a depth information of a certain region in the current frame image of the video stream relative to the previous frame. The degree of change in the depth information of the corresponding area in the image;

S4, for the region where the depth information changes in the overlap region, the image is spliced only once according to the acquired depth information; and for the region where the depth information changes greatly in the overlap region, frame-by-frame image splicing is performed according to the acquired depth information;

S5 affixes the spliced area with small change of depth information and the area where the depth information changes greatly, and generates the spliced overlapping area image; pastes the overlapping area of the panoramic video image and the non-intersecting overlapping area to generate the spliced panoramic video. image.

In the splicing method of the panoramic video image according to the present invention, obtaining the depth information of the overlapping area in each panoramic video image refers to: obtaining depth information of the overlapping area in each frame of the panoramic video image from the plurality of angles by the depth camera.

In the method for splicing a panoramic video image according to the present invention, the obtaining of the depth information includes: real-time acquiring depth information of a certain point or a certain area in the panoramic video image from two perspectives, and detecting two times in a certain time slot. When the depth information acquired by the viewing angles is the same, the connected depth information detected at this time is used as the depth information value of a certain point or a certain area in the panoramic video image.

In the splicing method of the panoramic video image according to the present invention, the image splicing refers to: acquiring image data of multiple depth levels from the corresponding panoramic video image overlapping area according to the depth information of the overlapping area in each panoramic video image. And splicing between image data for image data of the same depth level.

In the splicing method of the panoramic video image according to the present invention, the area where the depth information changes little means that the degree of change of the depth information of the area on the current frame image relative to the area corresponding to the previous frame image is less than a set threshold; The area where the depth information changes greatly means that the depth information of the area on the current image relative to the area corresponding to the previous frame image is greater than a set threshold.

The invention also provides a panoramic camera comprising:

a photographing module, configured to obtain a panoramic video image, the photographing module includes a plurality of two-dimensional cameras disposed in different directions of the same circumference, and an effective image capturing space captured by the adjacent two-dimensional camera has an edge overlapping overlapping area;

a depth information acquiring module, configured to acquire depth information of an overlapping area in the panoramic video image; the depth information is used to represent a spatial position of each pixel in the panoramic video image with respect to the shooting position;

An image detecting module, configured to detect a degree of change of depth information of the overlapping area in the panoramic video image, and divide the overlapping area into an area with small change of depth information and an area with large change of depth information according to the degree of change of the depth information; The degree of change of the information is the degree of change of the depth information of an area in the current frame image of the video stream relative to the depth information of the corresponding area in the previous frame image;

An image splicing module is configured to perform image splicing according to the acquired depth information in an area where the depth information changes in the overlapping area; and perform a frame-by-frame image according to the acquired depth information in the area where the depth information changes greatly in the overlapping area. Splicing; the image splicing refers to acquiring image data of a plurality of depth levels from corresponding overlapping regions of the panoramic video image according to the depth information, and performing splicing between the image data of the image data of the same depth level;

The image synthesizing module is configured to paste the area where the depth information changed after the splicing and the area where the depth information changes greatly, to generate the spliced overlapping area image; and paste the overlapping area of the panoramic video image and the non-intersecting overlapping area to generate Stitched panoramic video image.

In the panoramic camera of the present invention, the depth information acquisition module includes a plurality of depth cameras that acquire depth information of overlapping regions in each frame of panoramic video images from a plurality of perspectives.

In the panoramic camera of the present invention, the depth information acquiring module acquires depth information of a certain point or a certain area in the overlapping area of the panoramic video image in real time from two perspectives respectively, and when two time angles are detected in a certain time slot, When the depth information is the same, the same depth information detected at this time is used as the depth information value of a certain point or a certain area in the above-mentioned panoramic video image.

In the panoramic camera of the present invention, the image detecting module sets an area of the current frame image with respect to a region where the depth information of the region corresponding to the previous frame image is less than a set threshold, and the depth information changes little. The area is set to an area in which the depth information of the area on the current image with respect to the depth information of the area corresponding to the previous frame image is greater than the set threshold is set as the area where the depth information changes greatly.

In the panoramic camera of the present invention, the image splicing module acquires image data of a plurality of depth levels from the corresponding panoramic video image overlapping area according to the depth information of the overlapping area in each panoramic video image and the same depth level The image data is spliced between the image data to effect the image stitching.

Beneficial effect

Compared with the prior art, the splicing method of the panoramic video image provided by the present invention and the panoramic camera divide the overlapping area into a region with small change of depth information and a region with large change of depth information according to the degree of change of the depth information of the frame before and after the video stream, and The splicing is performed only on the area where the depth information changes little, the information data processing amount of the device is reduced, and the splicing efficiency of the overlapping area is improved. The depth information obtaining step provided by the present invention can quickly and accurately obtain a certain point or a certain point in the stereo image. The depth information of an area provides a solid foundation for obtaining high-resolution, seamless panoramic video images in the present invention; in addition, the method and apparatus provided by the present invention are simple and convenient to use, and reduce the complexity of panoramic video image stitching and improve The splicing efficiency of panoramic video images has the characteristics of fast and real-time splicing, which makes the device have a better user experience.

DRAWINGS

The present invention will be further described below in conjunction with the accompanying drawings and embodiments, in which:

1 is a flowchart of a method for splicing a panoramic video image according to an embodiment of the present invention;

[Correct according to Rule 91 02.03.2018]
2 is a schematic structural diagram of depth information acquisition according to an embodiment of the present invention;

FIG. 3 is a schematic structural diagram of a panoramic camera according to an embodiment of the present invention.

Embodiments of the invention

The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

As shown in FIG. 1 , this embodiment provides a method for splicing a panoramic video image, including the following steps.

S1 acquires a plurality of panoramic video images, and the panoramic video image is composed of a plurality of stereoscopic images captured in different directions of the same circumference, and the effective image capturing space of the adjacent stereoscopic images has an edge overlapping overlapping region.

The panoramic video image in this embodiment is captured by a panoramic camera, which may be a video frame with moving image information (such as a character moving in a shooting environment) or a relatively still video frame (environmental image). The panoramic camera includes a plurality of two-dimensional cameras arranged in different directions of the same circumference. The images captured by the plurality of two-dimensional cameras are generally panoramic images with an angle greater than 180 degrees, and in order to ensure the integrity of the panoramic images, adjacent different orientations There is an edge overlap overlapping area in the effective image capturing space captured by the 2D camera. The depth information of the same object or person whose depth of field displacement is obtained by the two sets of two-dimensional cameras respectively on the panoramic camera may show a difference.

S2 acquires depth information of an overlapping area in each panoramic video image, and the depth information is used to indicate a spatial position of each pixel in the panoramic video image with respect to the shooting position.

Obtaining the depth information of the overlapping area in each panoramic video image refers to: obtaining depth information of the overlapping area in each frame of the panoramic video image from the plurality of angles by the depth camera. In particular, for a potential person or object that generates a relative depth direction displacement in space, and some objects of the character's own spatial motion, gesture change, hand grip and movement, it will generate a range within the camera range of the panoramic camera. Depth of field depth information threshold point, depth information threshold point has the same depth of field feature for two adjacent two-dimensional cameras. Therefore, the embodiment provides a step of acquiring the depth information: the depth information of a certain point or a certain area in the panoramic video image is obtained by the two depth cameras from two perspectives in real time, and two angles of view are detected when a certain time slot is detected. When the depth information is the same, the same depth information detected at this time is used as the depth information value of a certain point or a certain area in the panoramic video image, that is, the depth information threshold point. As shown in FIG. 2, the movement trajectory of the person or object displaced by the outside relative to the depth direction is random, and the real-time depth information of the person or object randomly moving in the panoramic camera shooting environment is obtained by the above two adjacent depths. The camera obtains the real-time depth information of the person or object that is displaced in the depth direction in real time. When the depth information acquired by the adjacent depth cameras is consistent in the same time slot, the same depth information detected by the camera is used as the depth. Information threshold point. It should be noted that the above description is only for convenience of explaining the present embodiment, and the process of obtaining the depth information threshold point for a certain sampling point on the displaced person or object is generally described, for a person or an object in an overall movement. Preferably, the plurality of sampling points may be selected to perform the depth information threshold point obtaining operation of each sampling point at the same time, and the relatively centered depth information threshold point is used as the final depth information threshold point. For the display effect, the farther depth information shows that the depth of field information does not reflect the depth of field. Therefore, after acquiring the depth information threshold point, the depth will be acquired in real time by the adjacent two sets of depth cameras. The information is compared with the depth information threshold points, and one of the better ones is used as a reference for subsequent image stitching. Through this method, the depth information of a certain point or a certain area can be obtained simply and at low cost, and has good precision and accuracy.

S3 divides the depth information into a region where the depth information changes little and a region where the depth information changes greatly according to the degree of change of the depth information of the overlapping region; the depth information changes to a depth information of a certain region in the current frame image of the video stream relative to the previous frame. The degree of change in the depth information of the corresponding area in the image.

The area where the depth information changes little is that the depth information of the area on the current frame image is smaller than the set threshold value with respect to the area corresponding to the previous frame image; the area where the depth information changes greatly refers to the area on the current image relative to the upper part. The degree of change of the depth information of the area corresponding to one frame of image is greater than the set threshold. Specifically, the area where the depth information changes little generally refers to: for a general character indoor scene, that is, a fixedly placed furniture in the scene, and a fixed-position electrical device (eg, a light source, a large-screen display device, etc.), due to The placement of the above objects in such areas is basically unchanged, and the depth information displayed by the depth camera is also substantially unchanged or has little change, so that overlapping regions in the panoramic video image can be obtained by means such as a depth camera. The areas where the depth information changes little are extracted in advance, and the image mosaic of the overlapping areas is separately performed. The area where the depth information changes greatly generally refers to a person or thing that generates a relative depth direction displacement in the space, and some space motions, gesture changes, and objects that are held and moved by the character itself. If the relative displacement of the captured partial image relative to the previous shooting of the camera is large, the depth information reflected in the shooting area before and after a time slot will change greatly. For example, the person moves relatively back and forth with respect to the camera, and at the same time, different cameras are used. The people in the captured image are not in the same depth information, and different depth/disparity is generated. When the image is stitched, the image of the person and the non-person image are obtained from the overlapping regions of the corresponding video image according to the depth information of each video image. The non-person image is a fixed environment image in which the depth information in the overlapping area of the panoramic video image does not change substantially or changes little, and the character image is a person or object displacement image having dynamically variable depth information in the overlapping area of the panoramic video image; or The depth information of each video image acquires a background video image and a foreground image from the corresponding video image, the background video image is an area where the depth information changes little, and the foreground image is an area where the depth information changes greatly.

For the region where the depth information changes in the overlap region, the image is spliced only once according to the acquired depth information; and for the region where the depth information changes greatly in the overlap region, the frame-by-frame image splicing is performed according to the acquired depth information.

The image splicing refers to acquiring image data of a plurality of depth gradations from the corresponding panoramic video image overlapping area according to the depth information of the overlapping area in each panoramic video image, and performing splicing between the image data on the image data of the same depth level. In the specific implementation process, for the character image and the non-person image in the overlapping area of the panoramic video image, only the non-person image data is spliced once to generate non-person stitching image data; the image data of the person is stitched frame by frame to generate the character stitching. Image data; performing composite image processing on the stitched image data of the stitched image of each frame and the non-person stitching image data to generate an overlapped area image in the stitched panoramic video image; or in the overlapping area of the panoramic video image The acquired background image data is spliced once to generate background spliced image data; the acquired foreground image data is spliced frame by frame to generate foreground spliced image data; and the spliced image data of each frame of the spliced panoramic video image is synthesized to The background stitches the image data to generate an image of the overlap region in the stitched panoramic video image.

In this embodiment, the depth information of the current frame image in the panoramic video stream is compared with the depth information of the previous frame image, and only the region where the depth information changes in the overlapping region of the panoramic video image is frame-by-frame spliced, and the depth information changes in the overlapping region. The small area is spliced only once and is pasted with the area where the depth information after splicing in each panoramic video image changes greatly to form a spliced overlapping area image, and then the overlapping area of the panoramic video image and the non-intersecting overlapping area are pasted to generate Compared with the traditional stitching mode, the stitched panoramic video image omits the frame-by-frame stitching of the small area of the depth information change in the panoramic video image, reduces the complexity of the stitching of the panoramic video image, improves the stitching efficiency of the panoramic video image, and has a fast The characteristics of real-time stitching.

As shown in FIG. 3 , the present invention further provides a panoramic camera including a support frame and a housing disposed on the support frame, and further includes a shooting module disposed around the housing, a depth information acquiring module, and a housing disposed inside the housing. The image signal processor is electrically connected to the image capturing module and the depth information acquiring module, the shooting module is configured to capture the panoramic video image around the casing, and the depth information acquiring module is configured to acquire the depth of the panoramic video image captured by the shooting module. Information, the image signal processor is configured to detect the degree of change of the depth information of the panoramic video image and perform image stitching and image synthesis according to the degree of change of the depth information. The shooting module includes a plurality of two-dimensional cameras arranged in different directions of the same circumference, and the effective image capturing space captured by the adjacent two-dimensional cameras has an edge overlapping overlapping area. The depth information acquisition module includes a plurality of depth cameras that acquire depth information of overlapping regions in each frame of the panoramic video image from a plurality of perspectives. The embodiment further includes a power module for supplying power to the imaging module, the depth information acquiring module, and the image signal processor, and the power module adopts a large-capacity lithium battery, so that the embodiment has better endurance capability. In order to enable the embodiment to send the picture to the external device in time, the embodiment further includes a communication module electrically connected to the image synthesis module, and the communication module is configured to use the image communication module to stitch the panoramic video image through mobile communication or WiFi. , Bluetooth, infrared, etc. are transmitted to other devices, such as mobile phones, computers, etc., so that users can share the photos at any time.

Specifically, the shooting module is configured to acquire a panoramic video image, and the shooting module includes a plurality of two-dimensional cameras disposed in different directions of the same circumference. To ensure the integrity of the panoramic image, the effective image capturing space captured by the adjacent two-dimensional camera has an edge. Cross-overlapping areas, images taken by multiple two-dimensional cameras are typically panoramic video images with angles greater than 180 degrees. a depth information acquiring module, configured to acquire depth information of an overlapping area in the panoramic video image; the depth information is used to represent a spatial position of each pixel in the panoramic video image with respect to the shooting position; the depth information acquiring module includes multiple depth cameras, and more The depth cameras acquire depth information of overlapping regions in each frame of the panoramic video image from multiple perspectives. For a potential person or object that produces a relative depth direction displacement in space, as well as some of the character's own spatial motion, gesture changes, hand-held and moving objects, it will generate a depth-of-field depth information within the panoramic camera's imaging range. Threshold point, depth information threshold point has the same depth of field feature for two adjacent two-dimensional cameras. Specifically, the depth information acquisition module obtains the depth information of a certain point or a certain area in the overlapping area of the panoramic video image in real time from two perspectives. When the depth information acquired by the two perspectives is the same, the time is obtained. The detected same depth information is used as a depth information value of a certain point or a certain area in the above-mentioned panoramic video image. In addition, the depth information acquiring module may further select a plurality of sampling points to perform the depth information threshold point obtaining operation of each sampling point at a time, and use the relatively centered depth information threshold point as the final depth information threshold point. In this embodiment, a stereo camera that integrates the camera module and the depth information acquisition module is used. The stereo camera uses a left eye image acquisition unit and a right eye image acquisition unit to simultaneously acquire a series of left eye images and a series of corresponding images. The right eye image is displayed, and each pixel in the left eye image and the corresponding right eye image generates a depth information corresponding to each of the same pixels in the left eye image and the right eye image, and the stereo camera further uses the image processor according to the corresponding The depth information of each of the same pixels in the left eye image and the right eye image produces a stereoscopic panoramic video image. The image signal processor includes: an image detection module, an image mosaic module, and an image synthesis module. The image detecting module is configured to detect a degree of change of the depth information of the overlapping area in the panoramic video image, and divide the overlapping area into a region with a small change in depth information and a region with a large change in depth information according to the degree of change of the depth information; the degree of change of the depth information is a video stream. The degree of change of the depth information of an area in the current frame image relative to the depth information of the corresponding area in the previous frame image; the image detection module changes the depth information of the area on the current frame image relative to the area corresponding to the previous frame image by less than The area of the set threshold is set to a region where the depth information changes little, and the area on the current image is changed to the depth information with respect to the area where the depth information of the area corresponding to the previous frame image is greater than the set threshold. Area. Specifically, the corresponding video image overlapping area may be divided into a person image and a non-person image according to the depth information of each video image, and the non-person image, that is, the fixed information in which the depth information in the overlapping area of the panoramic video image does not change substantially or has little change The environmental image, the character image is a person or object displacement image with dynamically variable depth information in the overlapping area of the panoramic video image; or the background video image and the foreground image are obtained from the corresponding video image according to the depth information of each video image, the background video The image is an area where the depth information changes little, and the foreground image is an area where the depth information changes greatly. The image splicing module is configured to perform image splicing according to the acquired depth information in an area where the depth information changes in the overlapping area is small; and to perform frame-by-frame image splicing according to the acquired depth information in the area where the depth information changes greatly in the overlapping area; The module acquires image data of a plurality of depth levels from the corresponding panoramic video image overlapping area according to the depth information of the overlapping area in each panoramic video image, and performs image stitching by performing image stitching between the image data of the same depth level image data. The image synthesizing module is configured to paste the area where the depth information changed after the splicing and the area where the depth information changes greatly, to generate the spliced overlapping area image; and paste the overlapping area of the panoramic video image and the non-intersecting overlapping area to generate Stitched panoramic video image.

In the specific implementation process, for the character image and the non-person image in the overlapping area of the panoramic video image, only the non-person image data is spliced once to generate non-person stitching image data; the image data of the person is stitched frame by frame to generate the character stitching. Image data; performing composite image processing on the stitched image data of the stitched image of each frame and the non-person stitching image data to generate an overlapped area image in the stitched panoramic video image; or in the overlapping area of the panoramic video image The acquired background image data is spliced once to generate background spliced image data; the acquired foreground image data is spliced frame by frame to generate foreground spliced image data; and the spliced image data of each frame of the spliced panoramic video image is synthesized to The background stitches the image data to generate an image of the overlap region in the stitched panoramic video image. Then, the overlapping area of each panoramic video image and the non-intersecting overlapping area are pasted to generate a spliced panoramic video image. The device can effectively reduce the complexity of splicing of panoramic video images, improve the splicing efficiency of panoramic video images, and has the characteristics of fast and real-time splicing.

The embodiments of the present invention have been described above with reference to the drawings, but the present invention is not limited to the specific embodiments described above, and the specific embodiments described above are merely illustrative and not restrictive, and those skilled in the art In the light of the present invention, many forms may be made without departing from the spirit and scope of the invention as claimed.

Claims

A method for splicing panoramic video images, characterized in that it comprises the following steps,

S1 acquires a plurality of panoramic video images, which are composed of a plurality of stereoscopic images captured in different directions of the same circumference, and an effective image capturing space of the adjacent stereoscopic images has an edge overlapping overlapping region;

S2 acquiring depth information of an overlapping area in each panoramic video image, where the depth information is used to indicate a spatial position of each pixel in the panoramic video image with respect to the shooting position;

S3 divides the depth information into a region where the depth information changes little and a region where the depth information changes greatly according to the degree of change of the depth information of the overlapping region; the depth information changes to a depth information of a certain region in the current frame image of the video stream relative to the previous frame. The degree of change in the depth information of the corresponding area in the image;

S4, for the region where the depth information changes in the overlap region, the image is spliced only once according to the acquired depth information; and for the region where the depth information changes greatly in the overlap region, frame-by-frame image splicing is performed according to the acquired depth information;

S5 affixes the spliced area with small change of depth information and the area where the depth information changes greatly, and generates the spliced overlapping area image; pastes the overlapping area of the panoramic video image and the non-intersecting overlapping area to generate the spliced panoramic video. image.
The splicing method of the panoramic video image according to claim 1, wherein the obtaining the depth information of the overlapping area in each panoramic video image refers to: obtaining the depth of the overlapping area in each panoramic video image from the plurality of viewing angles by the depth camera information.
The method for splicing a panoramic video image according to claim 1, wherein the obtaining of the depth information comprises: acquiring real-time information of a certain point or a certain area in the panoramic video image from two perspectives, when a certain time When the gap detects that the depth information acquired by the two angles is the same, the joint depth information detected at this time is used as the depth information value of a certain point or a certain area in the panoramic video image.
The splicing method of the panoramic video image according to claim 1, wherein the image splicing refers to: acquiring a plurality of depths from the overlapping area of the corresponding panoramic video image according to the depth information of the overlapping area in each panoramic video image. Hierarchical image data and splicing of image data between image data of the same depth level.
The method for splicing a panoramic video image according to claim 1, wherein the area where the depth information changes little is that the depth information of the area on the current frame image is smaller than the depth information of the area corresponding to the previous frame image. The threshold is changed; the region where the depth information changes greatly means that the extent of the depth information of the region on the current image relative to the region corresponding to the previous frame image is greater than a set threshold.
A panoramic camera, comprising:

a photographing module, configured to obtain a panoramic video image, the photographing module includes a plurality of two-dimensional cameras disposed in different directions of the same circumference, and an effective image capturing space captured by the adjacent two-dimensional camera has an edge overlapping overlapping area;

a depth information acquiring module, configured to acquire depth information of an overlapping area in the panoramic video image; the depth information is used to represent a spatial position of each pixel in the panoramic video image with respect to the shooting position;

An image detecting module, configured to detect a degree of change of depth information of the overlapping area in the panoramic video image, and divide the overlapping area into an area with small change of depth information and an area with large change of depth information according to the degree of change of the depth information; The degree of change of the information is the degree of change of the depth information of an area in the current frame image of the video stream relative to the depth information of the corresponding area in the previous frame image;

An image splicing module is configured to perform image splicing according to the acquired depth information in an area where the depth information changes in the overlapping area; and perform a frame-by-frame image according to the acquired depth information in the area where the depth information changes greatly in the overlapping area. Splicing; the image splicing refers to acquiring image data of a plurality of depth levels from corresponding overlapping regions of the panoramic video image according to the depth information, and performing splicing between the image data of the image data of the same depth level;

The image synthesizing module is configured to paste the area where the depth information changed after the splicing and the area where the depth information changes greatly, to generate the spliced overlapping area image; and paste the overlapping area of the panoramic video image and the non-intersecting overlapping area to generate Stitched panoramic video image.
The panoramic camera according to claim 6, wherein the depth information acquisition module comprises a plurality of depth cameras, and the plurality of depth cameras acquire depth information of overlapping regions in each frame of the panoramic video image from a plurality of perspectives.
The panoramic camera according to claim 6, wherein the depth information acquisition module acquires depth information of a certain point or a certain area in the overlapping area of the panoramic video image in real time from two perspectives, and detects two time slots in a certain time slot. When the depth information acquired by the viewing angles is the same, the same depth information detected at this time is used as the depth information value of a certain point or a certain area in the panoramic video image.
The panoramic camera according to claim 6, wherein the image detecting module sets an area of the current frame image with respect to a region where the depth information of the region corresponding to the previous frame image is less than a set threshold as In a region where the depth information changes little, an area in which the depth information of the area corresponding to the previous frame image is greater than the set threshold value is set as an area in which the depth information changes greatly.
The panoramic camera according to claim 6, wherein the image splicing module acquires image data of a plurality of depth levels from the corresponding panoramic video image overlapping area according to the depth information of the overlapping area in each panoramic video image and Image data of the same depth level is spliced between image data to achieve the image stitching.