[go: up one dir, main page]

CN107995421A - A panoramic camera and its image generation method, system, device, and storage medium - Google Patents

A panoramic camera and its image generation method, system, device, and storage medium Download PDF

Info

Publication number
CN107995421A
CN107995421A CN201711237215.5A CN201711237215A CN107995421A CN 107995421 A CN107995421 A CN 107995421A CN 201711237215 A CN201711237215 A CN 201711237215A CN 107995421 A CN107995421 A CN 107995421A
Authority
CN
China
Prior art keywords
image
images
original
fisheye
group
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201711237215.5A
Other languages
Chinese (zh)
Inventor
陈烨
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Goertek Optical Technology Co Ltd
Original Assignee
Weifang Goertek Electronics Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Weifang Goertek Electronics Co Ltd filed Critical Weifang Goertek Electronics Co Ltd
Priority to CN201711237215.5A priority Critical patent/CN107995421A/en
Publication of CN107995421A publication Critical patent/CN107995421A/en
Pending legal-status Critical Current

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60Control of cameras or camera modules
    • H04N23/698Control of cameras or camera modules for achieving an enlarged field of view, e.g. panoramic image capture
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00Geometric image transformations in the plane of the image
    • G06T3/40Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4038Image mosaicing, e.g. composing plane images from plane sub-images
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/50Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/14Picture signal circuitry for video frequency region
    • H04N5/21Circuitry for suppressing or minimising disturbance, e.g. moiré or halo
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/222Studio circuitry; Studio devices; Studio equipment
    • H04N5/262Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/265Mixing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20212Image combination
    • G06T2207/20221Image fusion; Image merging

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Stereoscopic And Panoramic Photography (AREA)
  • Image Processing (AREA)
  • Studio Devices (AREA)

Abstract

This application discloses a kind of panorama camera and its image generating method, system, equipment, storage medium, this method to include:Gather the real-time lighting intensity of current outside environment;Judge whether real-time lighting intensity is less than default intensity of illumination threshold value, if it is, gathering one group of original fish eye images corresponding with each fish eye lens respectively, obtain the corresponding original fish eye images of N groups;Wherein, the original fish eye images of multiframe are included in every group of original fish eye images;Fusion treatment is carried out to every group of original fish eye images respectively, is correspondingly made available N blending images;Splicing is carried out to the N blending images, obtains spliced panoramic picture.The application is in light environment than in the case of dark, the original fish eye images of corresponding multiframe are obtained for each fish eye lens, by the way that the original fish eye images of multiframe in every group of original fish eye images are merged, noise in image can be effectively inhibited, so as to effectively improve the picture quality of the panoramic picture of shooting at night.

Description

一种全景相机及其图像生成方法、系统、设备、存储介质A panoramic camera and its image generation method, system, device, and storage medium

技术领域technical field

本发明涉及全景相机技术领域,特别涉及一种全景相机及其图像生成方法、系统、设备、存储介质。The present invention relates to the technical field of panoramic cameras, in particular to a panoramic camera and an image generation method, system, device, and storage medium thereof.

背景技术Background technique

VR(即Virtual Reality)技术能够让使用者进入一个三维空间的沉浸虚拟世界,如同身历其境一般。随着技术的进步,三维立体全景影像技术的应用范围越来越广。全景拍摄技术指的是利用科学技术、专业设备将整个真实的场景映射成全景图像,从而将二维的平面图模拟为三维立体空间,实现虚拟现实浏览。同时,浏览者可以在全景空间里进行切换,自由的观看“天地万物”。全景摄影成为新兴增长领域,而全景相机是全景摄影的核心设备。VR (Virtual Reality) technology allows users to enter a three-dimensional immersive virtual world, just like being there. With the advancement of technology, the application range of three-dimensional panoramic image technology is becoming wider and wider. Panoramic shooting technology refers to the use of science and technology and professional equipment to map the entire real scene into a panoramic image, thereby simulating a two-dimensional plan into a three-dimensional space to realize virtual reality browsing. At the same time, viewers can switch in the panoramic space and watch "everything in the world" freely. Panoramic photography has become an emerging growth field, and panoramic cameras are the core equipment of panoramic photography.

全景相机,就是能够进行全景拍摄的摄影器材。除拍照外,还可以应用于动态影像的拍摄,也被称为全景摄像机。由于单个相机镜头的视角范围小于360°,所以全景相机至少配备两个镜头。与传统相机/智能手机的水平旋转拍摄不同,全景相机能够在水平和垂直方向同一时刻拍摄,再通过全景拼接软件处理,生成360°×180°的全景画面。A panoramic camera is a photographic device capable of panoramic shooting. In addition to taking pictures, it can also be applied to the shooting of dynamic images, also known as panoramic cameras. Since the viewing angle range of a single camera lens is less than 360°, the panoramic camera is equipped with at least two lenses. Different from the horizontal rotation shooting of traditional cameras/smart phones, the panoramic camera can shoot at the same time in the horizontal and vertical directions, and then process it through the panoramic stitching software to generate a 360°×180° panoramic picture.

目前典型的全景相机是利用鱼眼光学镜头来进行全景图像拍摄的设备,一次摄取足够大的视场角的图像,其原理是依据仿生学(鱼眼构造),经过物理光学的球面镜透射以及折射,一次性地对水平360度,垂直180度的信息进行成像。At present, a typical panoramic camera is a device that uses a fisheye optical lens to shoot panoramic images. It captures images with a large enough field of view at one time. The principle is based on bionics (fisheye structure), through the transmission and refraction of the spherical mirror of physical optics. , to image information of 360 degrees horizontally and 180 degrees vertically at one time.

然而,当前全景相机全景图生成算法在某些情况下无法保证拍摄得到的全景图像的质量,特别是在夜间拍摄时,很难满足光照充足的条件,拍摄的低亮度全景图像普遍具有很低的对比度,夹杂着大量的噪声,人眼视觉感受较差,尽管相机传感器硬件已经有了飞速的发展和进步,但是由于硬件尺寸以及成本的限制,目前大部分全景相机设备的夜间拍照质量依然很难满足用户的需求和期望。However, the current panoramic camera panorama generation algorithm cannot guarantee the quality of the captured panoramic images in some cases, especially when shooting at night, it is difficult to meet the conditions of sufficient light, and the captured low-brightness panoramic images generally have very low Contrast, mixed with a lot of noise, the visual experience of the human eye is poor. Although the camera sensor hardware has been developed and improved rapidly, due to the limitation of hardware size and cost, the quality of night photos of most panoramic camera equipment is still difficult. Meet user needs and expectations.

综上所述可以看出,如何提升夜间拍摄的全景图像的图像质量是目前亟待解决的问题。From the above, it can be seen that how to improve the image quality of panoramic images taken at night is an urgent problem to be solved.

发明内容Contents of the invention

有鉴于此,本发明的目的在于提供一种全景相机及其图像生成方法、系统、设备、存储介质,能够有效地提升夜间拍摄的全景图像的图像质量。其具体方案如下:In view of this, the object of the present invention is to provide a panoramic camera and its image generation method, system, device, and storage medium, which can effectively improve the image quality of panoramic images taken at night. The specific plan is as follows:

第一方面,本发明公开了一种VR全景图像生成方法,应用于包括N个鱼眼镜头的全景相机,N为不小于2的整数;其中,所述方法包括:In the first aspect, the present invention discloses a method for generating a VR panoramic image, which is applied to a panoramic camera including N fisheye lenses, where N is an integer not less than 2; wherein, the method includes:

采集当前外界环境的实时光照强度;Collect the real-time light intensity of the current external environment;

判断实时光照强度是否小于预设光照强度阈值,如果是,则分别采集与每个鱼眼镜头对应的一组原始鱼眼图像,得到相应的N组原始鱼眼图像;其中,每组原始鱼眼图像中均包括多帧原始鱼眼图像;Determine whether the real-time light intensity is less than the preset light intensity threshold, and if so, collect a group of original fisheye images corresponding to each fisheye lens to obtain corresponding N groups of original fisheye images; wherein, each group of original fisheye The images all include multiple frames of original fisheye images;

分别对每组原始鱼眼图像进行融合处理,相应地得到N张融合图像;Perform fusion processing on each group of original fisheye images respectively, and obtain N fusion images accordingly;

对所述N张融合图像进行拼接处理,得到拼接后的全景图像。Stitching is performed on the N fusion images to obtain a stitched panoramic image.

可选的,所述分别采集与每个鱼眼镜头对应的一组原始鱼眼图像,得到相应的N组原始鱼眼图像的步骤,包括:Optionally, the step of collecting a group of original fisheye images corresponding to each fisheye lens to obtain corresponding N groups of original fisheye images includes:

确定实时光照强度的强度等级,得到目标强度等级;Determine the intensity level of the real-time light intensity to obtain the target intensity level;

利用预设的强度等级与每组原始鱼眼图像的采集数量之间的映射关系,确定出与所述目标强度等级对应的采集数量,得到目标采集数量;Using the mapping relationship between the preset intensity level and the acquisition quantity of each group of original fisheye images, determine the acquisition quantity corresponding to the target intensity level, and obtain the target acquisition quantity;

根据所述目标采集数量,分别采集与每个鱼眼镜头对应的一组原始鱼眼图像,得到相应的N组原始鱼眼图像;其中,每组原始鱼眼图像的图像帧数与所述目标采集数量相等。According to the number of target acquisitions, a group of original fisheye images corresponding to each fisheye lens are respectively collected to obtain corresponding N groups of original fisheye images; wherein, the number of image frames of each group of original fisheye images is the same as that of the target The number of collections is equal.

可选的,对每组原始鱼眼图像进行融合处理的过程,包括:Optionally, the process of fusing each group of original fisheye images includes:

从该组原始鱼眼图像筛选出一帧原始鱼眼图像作为参考帧图像;A frame of original fish-eye image is selected from the group of original fish-eye images as a reference frame image;

将该组原始鱼眼图像中的其他原始鱼眼图像均与所述参考帧图像进行坐标对齐,得到相应的对齐后图像序列;All other original fisheye images in the group of original fisheye images are coordinate-aligned with the reference frame image to obtain a corresponding aligned image sequence;

对所述对齐后图像序列中的所有图像进行加权融合,得到相应的一张融合图像。All the images in the aligned image sequence are weighted and fused to obtain a corresponding fused image.

可选的,所述从该组原始鱼眼图像筛选出一帧原始鱼眼图像作为参考帧图像的步骤,包括:Optionally, the step of selecting a frame of original fisheye image from the group of original fisheye images as a reference frame image includes:

分别计算该组原始鱼眼图像中每帧原始鱼眼图像的平均梯度,然后将平均梯度最大的一帧原始鱼眼图像作为参考帧图像。The average gradient of each frame of the original fisheye image in the group of original fisheye images is calculated respectively, and then the frame of the original fisheye image with the largest average gradient is used as the reference frame image.

可选的,所述对所述对齐后图像序列中的所有图像进行加权融合,得到相应的一张融合图像的步骤,包括:Optionally, the step of performing weighted fusion on all images in the aligned image sequence to obtain a corresponding fused image includes:

分别根据所述对齐后图像序列中每一帧图像与所述参考帧图像所对应的图像坐标之间的像素灰度差值的绝对值,确定所述对齐后图像序列中每一帧图像的加权系数;Determine the weight of each frame of image in the aligned image sequence according to the absolute value of the pixel grayscale difference between each frame of image in the aligned image sequence and the image coordinates corresponding to the reference frame image coefficient;

根据所述对齐后图像序列中每一帧图像的加权系数,对所述对齐后图像序列中的所有图像进行加权融合,得到相应的一张融合图像。All the images in the aligned image sequence are weighted and fused according to the weight coefficient of each frame of image in the aligned image sequence to obtain a corresponding fused image.

可选的,所述对所述N张融合图像进行拼接处理,得到拼接后的全景图像的步骤,包括:Optionally, the step of stitching the N fusion images to obtain the stitched panoramic image includes:

分别对每张融合图像进行平面展开,得到N张展开后图像;Carry out planar expansion on each fused image respectively to obtain N expanded images;

对所述N张展开后图像进行重合区域的配准处理,得到相应的配准结果;performing registration processing on overlapping regions of the N expanded images to obtain corresponding registration results;

根据所述配准结果,对所述N张展开后图像的重合区域进行融合处理,得到所述全景图像。According to the registration result, fusion processing is performed on overlapping regions of the N expanded images to obtain the panoramic image.

第二方面,本发明公开了一种VR全景图像生成系统,应用于包括N个鱼眼镜头的全景相机,N为不小于2的整数;其中,所述系统包括:In a second aspect, the present invention discloses a VR panoramic image generation system, which is applied to a panoramic camera including N fisheye lenses, where N is an integer not less than 2; wherein, the system includes:

光照强度采集模块,用于采集当前外界环境的实时光照强度;The light intensity acquisition module is used to collect the real-time light intensity of the current external environment;

光照强度判断模块,用于判断实时光照强度是否小于预设光照强度阈值;The light intensity judging module is used to judge whether the real-time light intensity is less than the preset light intensity threshold;

鱼眼图像采集模块,用于如果是,则分别采集与每个鱼眼镜头对应的一组原始鱼眼图像,得到相应的N组原始鱼眼图像;其中,每组原始鱼眼图像中均包括多帧原始鱼眼图像;The fisheye image acquisition module is used to collect a group of original fisheye images corresponding to each fisheye lens respectively to obtain corresponding N groups of original fisheye images; wherein, each group of original fisheye images includes Multi-frame original fisheye image;

鱼眼图像融合模块,用于分别对每组原始鱼眼图像进行融合处理,相应地得到N张融合图像;The fisheye image fusion module is used to fuse each group of original fisheye images respectively, and obtain N pieces of fusion images accordingly;

图像拼接模块,用于对所述N张融合图像进行拼接处理,得到拼接后的全景图像。An image splicing module, configured to splice the N fusion images to obtain a spliced panoramic image.

第三方面,本发明公开了一种VR全景图像生成设备,应用于包括N个鱼眼镜头的全景相机,N为不小于2的整数;其中,所述设备包括光照传感器、处理器和存储器;其中,所述处理器通过执行所述存储器中保存的VR全景图像生成程序实现前述公开的VR全景图像生成方法。In a third aspect, the present invention discloses a VR panoramic image generation device, which is applied to a panoramic camera including N fisheye lenses, where N is an integer not less than 2; wherein, the device includes an illumination sensor, a processor, and a memory; Wherein, the processor implements the VR panoramic image generation method disclosed above by executing the VR panoramic image generation program stored in the memory.

第四方面,本发明公开了一种全景相机,包括N个鱼眼镜头,其中,N为不小于2的整数,还包括前述公开的VR全景图像生成设备。In a fourth aspect, the present invention discloses a panoramic camera, which includes N fisheye lenses, where N is an integer not less than 2, and also includes the VR panoramic image generation device disclosed above.

第五方面,本发明公开了一种计算机可读存储介质,用于存储VR全景图像生成程序,其中,所述VR全景图像生成程序被处理器执行时实现前述公开的VR全景图像生成方法。In a fifth aspect, the present invention discloses a computer-readable storage medium for storing a VR panoramic image generation program, wherein, when the VR panoramic image generation program is executed by a processor, the aforementioned disclosed VR panoramic image generation method is realized.

可见,本发明中的全景相机包括N个鱼眼镜头,在外界环境的实时光照强度小于预设的光照强度阈值时,本发明将会分别采集与每个鱼眼镜头相对应的一组原始鱼眼图像,从而得到N组原始鱼眼图像,并且每组原始鱼眼图像中均包括多帧原始鱼眼图像。然后分别对每组原始鱼眼图像进行融合处理,并将融合后得到的N张图像进行拼接,从而得到拼接后的全景图像。由此可见,本发明在光照环境比较暗的情况下,针对每个鱼眼镜头均获取相应的多帧原始鱼眼图像,通过将每组原始鱼眼图像中的多帧原始鱼眼图像进行融合,可以有效地抑制图像中噪声,从而得到与上述N个鱼眼镜头分别一一对应的噪声较小的N张融合图像,由此使得后续拼接处理的全景图像的噪声得到有效地抑制,从而有效地提升了夜间拍摄的全景图像的图像质量。It can be seen that the panoramic camera in the present invention includes N fisheye lenses. When the real-time light intensity of the external environment is less than the preset light intensity threshold, the present invention will respectively collect a group of original fish corresponding to each fisheye lens. eye images, thereby obtaining N groups of original fish-eye images, and each group of original fish-eye images includes multiple frames of original fish-eye images. Then each group of original fisheye images is fused separately, and the fused N images are stitched together to obtain a stitched panoramic image. It can be seen that the present invention obtains corresponding multiple frames of original fish-eye images for each fish-eye lens when the lighting environment is relatively dark, and fuses the multiple frames of original fish-eye images in each group of original fish-eye images. , can effectively suppress the noise in the image, so as to obtain N fused images with less noise corresponding to the above N fisheye lenses, so that the noise of the panoramic image in the subsequent stitching process can be effectively suppressed, thus effectively Greatly improved the image quality of panoramic images captured at night.

附图说明Description of drawings

为了更清楚地说明本发明实施例或现有技术中的技术方案,下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本发明的实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据提供的附图获得其他的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only It is an embodiment of the present invention, and those skilled in the art can also obtain other drawings according to the provided drawings without creative work.

图1为本发明实施例公开的一种VR全景图像生成方法流程图;Fig. 1 is a flow chart of a method for generating a VR panoramic image disclosed in an embodiment of the present invention;

图2为本发明实施例公开的一种具体的VR全景图像生成方法流程图;FIG. 2 is a flow chart of a specific method for generating a VR panoramic image disclosed in an embodiment of the present invention;

图3为本发明实施例公开的一种具体的VR全景图像生成方法流程图;FIG. 3 is a flow chart of a specific method for generating a VR panoramic image disclosed in an embodiment of the present invention;

图4为本发明实施例公开的一种VR全景图像生成系统结构示意图。FIG. 4 is a schematic structural diagram of a VR panoramic image generation system disclosed in an embodiment of the present invention.

具体实施方式Detailed ways

下面将结合本发明实施例中的附图,对本发明实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例仅仅是本发明一部分实施例,而不是全部的实施例。基于本发明中的实施例,本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例,都属于本发明保护的范围。The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

本发明实施例公开了一种VR全景图像生成方法,应用于包括N个鱼眼镜头的全景相机,上述N为不小于2的整数;参见图1所示,上述方法包括:The embodiment of the present invention discloses a method for generating a VR panoramic image, which is applied to a panoramic camera including N fisheye lenses, where N is an integer not less than 2; as shown in FIG. 1 , the method includes:

步骤S11:采集当前外界环境的实时光照强度。Step S11: Collect the real-time light intensity of the current external environment.

本实施例中,可以通过主动采集方式来采集当前外界环境的实时光照强度,也即是令全景相机在开机后便一直开启光照强度的采集功能。In this embodiment, the real-time light intensity of the current external environment can be collected through an active collection method, that is, the collection function of the light intensity can be enabled all the time after the panoramic camera is turned on.

另外,本实施例也可以采用半主动采集方式来采集当前外界环境的实时光照强度,也即是根据实时获取到的其他类型参数来判断是否需要开启光照强度的采集功能,如果是,则开启光照强度的采集功能,以获取当前外界环境的实时光照强度,其中,上述实时获取到的其他类型参数可以包括但不限于当前位置区域的时间参数。例如,如果当前实时获取到的时间参数为晚上8点,则可以根据该时间参数判定当前需要开启光照强度的采集功能,如果当前实时获取到的时间参数为早上8点,则可以根据该时间参数判定当前无需开启光照强度的采集功能。In addition, this embodiment can also use a semi-active collection method to collect the real-time light intensity of the current external environment, that is, judge whether it is necessary to enable the light intensity collection function according to other types of parameters obtained in real time, and if so, turn on the light intensity Intensity collection function to obtain real-time light intensity of the current external environment, wherein other types of parameters obtained in real time may include but not limited to time parameters of the current location area. For example, if the current time parameter obtained in real time is 8:00 p.m., it can be determined according to the time parameter that the collection function of light intensity needs to be enabled; It is determined that there is no need to enable the collection function of light intensity at present.

当然,本实施例也可以采用被动采集方式来采集当前外界环境的实时光照强度,也即是当获取到由预设的触发按钮在被按压的情况下所产生的触发信号,则可以开启光照强度的采集功能。Of course, this embodiment can also adopt a passive collection method to collect the real-time light intensity of the current external environment, that is, when the trigger signal generated by the preset trigger button is obtained, the light intensity can be turned on. collection function.

步骤S12:判断实时光照强度是否小于预设光照强度阈值,如果是,则分别采集与每个鱼眼镜头对应的一组原始鱼眼图像,得到相应的N组原始鱼眼图像;其中,每组原始鱼眼图像中均包括多帧原始鱼眼图像。Step S12: Determine whether the real-time light intensity is less than the preset light intensity threshold, and if so, collect a group of original fisheye images corresponding to each fisheye lens to obtain corresponding N groups of original fisheye images; wherein, each group Each original fisheye image includes multiple frames of original fisheye images.

本实施例中,实时光照强度小于预设光照强度阈值时,意味着当前的光照强度不够,会使得图像中出现对图像质量产生不良影响的噪声,这种情况下分别采集与每个鱼眼镜头对应的原始鱼眼图像,从而得到N组原始鱼眼图像,其中,每组原始鱼眼图像中均包括多帧原始鱼眼图像。可以理解的是,本实施例中任意两组原始鱼眼图像中的原始鱼眼图像的帧数可以相等,也可以不等。In this embodiment, when the real-time light intensity is less than the preset light intensity threshold, it means that the current light intensity is not enough, which will cause noise in the image that will adversely affect the image quality. corresponding original fish-eye images, thereby obtaining N groups of original fish-eye images, wherein each group of original fish-eye images includes multiple frames of original fish-eye images. It can be understood that, in this embodiment, the number of frames of the original fish-eye images in any two groups of original fish-eye images may be equal or unequal.

需要说明的是,本实施例中,上述预设光照强度阈值可以由用户根据实际情况需要进行相应的具体设定。也即,在上述步骤S12之前还需获取用户通过预设的阈值设定接口输入的光照强度阈值。当然,本实施例中,上述预设光照强度阈值也可以由全景相机根据实际经验值进行自动设定。It should be noted that, in this embodiment, the aforementioned preset light intensity threshold may be specifically set by the user according to actual needs. That is, before the above step S12, it is necessary to obtain the light intensity threshold input by the user through the preset threshold setting interface. Of course, in this embodiment, the aforementioned preset light intensity threshold may also be automatically set by the panoramic camera according to actual experience values.

步骤S13:分别对每组原始鱼眼图像进行融合处理,相应地得到N张融合图像。Step S13: Carry out fusion processing on each group of original fisheye images respectively, and obtain N fusion images accordingly.

本实施例中,在对每组原始鱼眼图像中的多帧原始鱼眼图像进行融合之后,可以有效抑制最终融合图像中的图像噪声,使得每组原始鱼眼图像对应的融合图像的图像噪声大大少于该组原始鱼眼图像中的每帧图像。In this embodiment, after the multi-frame original fisheye images in each group of original fisheye images are fused, the image noise in the final fused image can be effectively suppressed, so that the image noise of the fused image corresponding to each group of original fisheye images Significantly less than each frame in the set of raw fisheye images.

步骤S14:对上述N张融合图像进行拼接处理,得到拼接后的全景图像。Step S14: Perform splicing processing on the above N fusion images to obtain a spliced panoramic image.

本实施例中,对上述N张融合图像进行拼接处理,得到拼接后的全景图像的步骤,具体可以包括:In this embodiment, the step of splicing the above N fusion images to obtain the spliced panoramic image may specifically include:

分别对每张融合图像进行平面展开,得到N张展开后图像;对上述N张展开后图像进行重合区域的配准处理,得到相应的配准结果;根据该配准结果,对上述N张展开后图像的重合区域进行融合处理,得到全景图像。Carry out planar expansion on each fused image respectively to obtain N expanded images; perform registration processing on overlapping areas of the above N expanded images to obtain corresponding registration results; according to the registration results, expand the above N images The overlapping areas of the subsequent images are fused to obtain a panoramic image.

可见,本发明实施例中的全景相机包括N个鱼眼镜头,在外界环境的实时光照强度小于预设的光照强度阈值时,本发明实施例将会分别采集与每个鱼眼镜头相对应的一组原始鱼眼图像,从而得到N组原始鱼眼图像,并且每组原始鱼眼图像中均包括多帧原始鱼眼图像。然后分别对每组原始鱼眼图像进行融合处理,并将融合后得到的N张图像进行拼接,从而得到拼接后的全景图像。由此可见,本发明实施例在光照环境比较暗的情况下,针对每个鱼眼镜头均获取相应的多帧原始鱼眼图像,通过将每组原始鱼眼图像中的多帧原始鱼眼图像进行融合,可以有效地抑制图像中噪声,从而得到与上述N个鱼眼镜头分别一一对应的噪声较小的N张融合图像,由此使得后续拼接处理的全景图像的噪声得到有效地抑制,从而有效地提升了夜间拍摄的全景图像的图像质量。It can be seen that the panoramic camera in the embodiment of the present invention includes N fisheye lenses. When the real-time light intensity of the external environment is less than the preset light intensity threshold, the embodiment of the present invention will respectively collect the light corresponding to each fisheye lens. A group of original fish-eye images, thereby obtaining N groups of original fish-eye images, and each group of original fish-eye images includes multiple frames of original fish-eye images. Then each group of original fisheye images is fused separately, and the fused N images are stitched together to obtain a stitched panoramic image. It can be seen that, in the embodiment of the present invention, when the lighting environment is relatively dark, corresponding multi-frame original fish-eye images are obtained for each fish-eye lens, by combining the multi-frame original fish-eye images in each group of original fish-eye images Fusion can effectively suppress the noise in the image, so as to obtain N fused images with less noise corresponding to the above N fisheye lenses, thereby effectively suppressing the noise of the panoramic image in the subsequent splicing process, Therefore, the image quality of the panoramic image captured at night is effectively improved.

在前述实施例的基础上,本发明实施例公开了一种具体的VR全景图像生成方法,参见图2所示,该方法包括:On the basis of the foregoing embodiments, the embodiment of the present invention discloses a specific method for generating a VR panoramic image, as shown in FIG. 2 , the method includes:

步骤S21:采集当前外界环境的实时光照强度。Step S21: Collect the real-time light intensity of the current external environment.

步骤S22:判断实时光照强度是否小于预设光照强度阈值,如果是,则确定实时光照强度的强度等级,得到目标强度等级。Step S22: judging whether the real-time light intensity is less than the preset light intensity threshold, and if so, determining the intensity level of the real-time light intensity to obtain the target intensity level.

本实施例中,当实时光照强度小于预设光照强度阈值时,确定实时光照强度的强度等级。可以理解的是,在上述步骤S22之前,需要先确定出光照强度的数值大小范围与强度等级之间的对应关系,根据该对应关系以及实时光照强度所处的数值大小范围,可以确定出相应的强度等级。本实施例中,强度等级越高,则可说明相应的实时光照强度的数值越大,相反,强度等级越低,则可说明相应的实时光照强度的数值越小。In this embodiment, when the real-time light intensity is less than the preset light intensity threshold, the intensity level of the real-time light intensity is determined. It can be understood that, before the above step S22, it is necessary to determine the corresponding relationship between the numerical range of the light intensity and the intensity level, and according to the corresponding relationship and the numerical value range of the real-time light intensity, the corresponding power level. In this embodiment, the higher the intensity level, the greater the value of the corresponding real-time light intensity; on the contrary, the lower the intensity level, the smaller the value of the corresponding real-time light intensity.

步骤S23:利用预设的强度等级与每组原始鱼眼图像的采集数量之间的映射关系,确定出与目标强度等级对应的采集数量,得到目标采集数量。Step S23: Using the mapping relationship between the preset intensity level and the acquisition quantity of each group of original fisheye images, determine the acquisition quantity corresponding to the target intensity level, and obtain the target acquisition quantity.

可见,本实施例中,在采集任一鱼眼镜头对应的原始鱼眼图像时,所采集的原始鱼眼图像的帧数量与当前实时光照强度的强度等级是相互对应的,也即不同的强度等级对应于不同的采集数量。可以理解的是,实时光照强度的强度等级与上述目标采集数量之间呈反相关关系,也即,强度等级越低,说明此时的图像噪声越大,由此可将上述目标采集数量设为越大的数值。并且需要进一步说明的是,上述目标采集数量不小于2。It can be seen that in this embodiment, when collecting the original fisheye image corresponding to any fisheye lens, the number of frames of the collected original fisheye image and the intensity level of the current real-time light intensity correspond to each other, that is, different intensity The levels correspond to different acquisition quantities. It can be understood that there is an inverse correlation between the intensity level of the real-time light intensity and the above-mentioned number of target acquisitions, that is, the lower the intensity level, the greater the image noise at this time, so the above-mentioned number of target acquisitions can be set to The larger the value. And it needs to be further explained that the above-mentioned number of target acquisitions is not less than 2.

步骤S24:根据目标采集数量,分别采集与每个鱼眼镜头对应的一组原始鱼眼图像,得到相应的N组原始鱼眼图像;其中,每组原始鱼眼图像的图像帧数与目标采集数量相等。Step S24: Collect a group of original fisheye images corresponding to each fisheye lens according to the number of target acquisitions, and obtain corresponding N groups of original fisheye images; wherein, the number of image frames of each group of original fisheye images is the same as that of the target acquisition The quantity is equal.

也即,在确定出上述目标采集数量之后,针对每一鱼眼镜头,均采集n帧原始鱼眼图像,其中该n值与上述目标采集数量相等。That is, after the above-mentioned number of target acquisitions is determined, for each fisheye lens, n frames of original fish-eye images are collected, wherein the value of n is equal to the above-mentioned number of target acquisitions.

步骤S25:分别对每组原始鱼眼图像进行融合处理,相应地得到N张融合图像。Step S25: performing fusion processing on each group of original fisheye images respectively, and correspondingly obtaining N fusion images.

本实施例中,对每组原始鱼眼图像进行融合处理的过程,具体可以包括:In this embodiment, the process of performing fusion processing on each group of original fisheye images may specifically include:

从该组原始鱼眼图像筛选出一帧原始鱼眼图像作为参考帧图像;将该组原始鱼眼图像中的其他原始鱼眼图像均与参考帧图像进行坐标对齐,得到相应的对齐后图像序列;对对齐后图像序列中的所有图像进行加权融合,得到相应的一张融合图像。Select an original fisheye image from the group of original fisheye images as a reference frame image; coordinate alignment of other original fisheye images in the group of original fisheye images with the reference frame image, and obtain the corresponding aligned image sequence ; Perform weighted fusion on all the images in the aligned image sequence to obtain a corresponding fused image.

另外,上述从该组原始鱼眼图像筛选出一帧原始鱼眼图像作为参考帧图像的步骤,具体可以包括:In addition, the above-mentioned step of selecting a frame of original fish-eye image from the group of original fish-eye images as a reference frame image may specifically include:

分别计算该组原始鱼眼图像中每帧原始鱼眼图像的平均梯度,然后将平均梯度最大的一帧原始鱼眼图像作为参考帧图像。The average gradient of each frame of the original fisheye image in the group of original fisheye images is calculated respectively, and then the frame of the original fisheye image with the largest average gradient is used as the reference frame image.

进一步的,上述对对齐后图像序列中的所有图像进行加权融合,得到相应的一张融合图像的步骤,具体可以包括:Further, the above step of performing weighted fusion on all the images in the aligned image sequence to obtain a corresponding fused image may specifically include:

分别根据对齐后图像序列中每一帧图像与参考帧图像所对应的图像坐标之间的像素灰度差值的绝对值,确定对齐后图像序列中每一帧图像的加权系数;根据对齐后图像序列中每一帧图像的加权系数,对对齐后图像序列中的所有图像进行加权融合,得到相应的一张融合图像。Determine the weighting coefficient of each frame image in the aligned image sequence according to the absolute value of the pixel grayscale difference between each frame image in the aligned image sequence and the image coordinates corresponding to the reference frame image; The weighting coefficient of each frame of image in the sequence is weighted and fused to all the images in the aligned image sequence to obtain a corresponding fused image.

步骤S26:对上述N张融合图像进行拼接处理,得到拼接后的全景图像。Step S26: Perform splicing processing on the above N fusion images to obtain a spliced panoramic image.

本实施例中,对上述N张融合图像进行拼接处理,得到拼接后的全景图像的步骤,具体可以包括:In this embodiment, the step of splicing the above N fusion images to obtain the spliced panoramic image may specifically include:

分别对每张融合图像进行平面展开,得到N张展开后图像;对上述N张展开后图像进行重合区域的配准处理,得到相应的配准结果;根据该配准结果,对上述N张展开后图像的重合区域进行融合处理,得到全景图像。Carry out planar expansion on each fused image respectively to obtain N expanded images; perform registration processing on overlapping areas of the above N expanded images to obtain corresponding registration results; according to the registration results, expand the above N images The overlapping areas of the subsequent images are fused to obtain a panoramic image.

在前述实施例的基础上,本发明实施例公开了一种更加具体的VR全景图像生成方法,应用于包括2个鱼眼镜头的全景相机;参见图3所示,该方法包括:On the basis of the foregoing embodiments, the embodiment of the present invention discloses a more specific method for generating a VR panoramic image, which is applied to a panoramic camera including two fisheye lenses; as shown in FIG. 3 , the method includes:

步骤S301:采集当前外界环境的实时光照强度。Step S301: Collect the real-time light intensity of the current external environment.

步骤S302:判断实时光照强度是否小于预设光照强度阈值,如果是,则分别采集与每个鱼眼镜头对应的一组原始鱼眼图像,得到相应的2组原始鱼眼图像;其中,每组原始鱼眼图像中均包括多帧原始鱼眼图像。Step S302: Determine whether the real-time light intensity is less than the preset light intensity threshold, and if so, collect a set of original fisheye images corresponding to each fisheye lens to obtain two corresponding sets of original fisheye images; wherein, each set Each original fisheye image includes multiple frames of original fisheye images.

步骤S303:计算第一组原始鱼眼图像中每帧原始鱼眼图像的平均梯度,计算第二组原始鱼眼图像中每帧原始鱼眼图像的平均梯度。Step S303: Calculate the average gradient of each frame of the original fisheye image in the first group of original fisheye images, and calculate the average gradient of each frame of the original fisheye image in the second group of original fisheye images.

步骤S304:将第一组原始鱼眼图像中平均梯度最大的一帧原始鱼眼图像作为第一参考帧图像,将第二组原始鱼眼图像中平均梯度最大的一帧原始鱼眼图像作为第二参考帧图像。Step S304: Take the frame of original fisheye image with the largest average gradient in the first group of original fisheye images as the first reference frame image, and use the frame of original fisheye image with the largest average gradient in the second group of original fisheye images as the second frame of original fisheye image Two reference frame images.

步骤S305:将第一组原始鱼眼图像中的其他原始鱼眼图像均与第一参考帧图像进行坐标对齐,得到第一对齐后图像序列;将第二组原始鱼眼图像中的其他原始鱼眼图像均与第二参考帧图像进行坐标对齐,得到第二对齐后图像序列。Step S305: Align other original fisheye images in the first group of original fisheye images with the first reference frame image to obtain the first aligned image sequence; align other original fisheye images in the second group of original fisheye images The eye images are coordinate-aligned with the second reference frame image to obtain a second aligned image sequence.

步骤S306:根据第一对齐后图像序列中每一帧图像与第一参考帧图像所对应的图像坐标之间的像素灰度差值的绝对值,确定对第一齐后图像序列中每一帧图像的加权系数;根据第二对齐后图像序列中每一帧图像与第二参考帧图像所对应的图像坐标之间的像素灰度差值的绝对值,确定对第二齐后图像序列中每一帧图像的加权系数。Step S306: According to the absolute value of the pixel grayscale difference between each frame image in the first aligned image sequence and the image coordinates corresponding to the first reference frame image, determine the alignment for each frame in the first aligned image sequence The weighting coefficient of the image; according to the absolute value of the pixel grayscale difference between each frame image in the second aligned image sequence and the corresponding image coordinates of the second reference frame image, it is determined for each frame image in the second aligned image sequence The weighting coefficient of an image frame.

步骤S307:根据第一对齐后图像序列中每一帧图像的加权系数,对第一对齐后图像序列中的所有图像进行加权融合,得到第一张融合图像;根据第二对齐后图像序列中每一帧图像的加权系数,对第二对齐后图像序列中的所有图像进行加权融合,得到第二张融合图像。Step S307: According to the weight coefficient of each frame image in the first aligned image sequence, perform weighted fusion on all the images in the first aligned image sequence to obtain the first fused image; The weighting coefficient of one frame of images is used to perform weighted fusion on all the images in the second aligned image sequence to obtain the second fused image.

步骤S308:分别对每张融合图像进行平面展开,得到2张展开后图像;Step S308: performing planar expansion on each fused image respectively to obtain two expanded images;

步骤S309:对上述2张展开后图像进行重合区域的配准处理,得到相应的配准结果;Step S309: Perform registration processing on the overlapping areas of the above two expanded images to obtain corresponding registration results;

步骤S310:根据配准结果,对上述2张展开后图像的重合区域进行融合处理,得到全景图像。Step S310: According to the registration result, fusion processing is performed on the overlapped regions of the two expanded images to obtain a panoramic image.

相应的,本发明实施例还公开了一种VR全景图像生成系统,应用于包括N个鱼眼镜头的全景相机,N为不小于2的整数;其中,参见图4所示,该系统包括:Correspondingly, the embodiment of the present invention also discloses a VR panoramic image generation system, which is applied to a panoramic camera including N fisheye lenses, where N is an integer not less than 2; wherein, as shown in FIG. 4 , the system includes:

光照强度采集模块11,用于采集当前外界环境的实时光照强度;The light intensity acquisition module 11 is used to collect the real-time light intensity of the current external environment;

光照强度判断模块12,用于判断实时光照强度是否小于预设光照强度阈值;Illumination intensity judging module 12, for judging whether the real-time illumination intensity is less than the preset illumination intensity threshold;

鱼眼图像采集模块13,用于如果是,则分别采集与每个鱼眼镜头对应的一组原始鱼眼图像,得到相应的N组原始鱼眼图像;其中,每组原始鱼眼图像中均包括多帧原始鱼眼图像;Fisheye image acquisition module 13, for if yes, collect a group of original fisheye images corresponding to each fisheye lens respectively, obtain corresponding N groups of original fisheye images; Wherein, in each group of original fisheye images Including multiple frames of original fisheye images;

鱼眼图像融合模块14,用于分别对每组原始鱼眼图像进行融合处理,相应地得到N张融合图像;The fisheye image fusion module 14 is used to fuse each group of original fisheye images respectively, and obtain N pieces of fusion images accordingly;

图像拼接模块15,用于对上述N张融合图像进行拼接处理,得到拼接后的全景图像。The image stitching module 15 is configured to stitch the above N fusion images to obtain stitched panoramic images.

关于上述各个模块更加具体的工作过程可以参考前述实施例中公开的相应内容,在此不再进行赘述。For the more specific working process of each of the above modules, reference may be made to the corresponding content disclosed in the foregoing embodiments, which will not be repeated here.

相应的,本发明还公开了一种VR全景图像生成设备,应用于包括N个鱼眼镜头的全景相机,N为不小于2的整数;其中,设备包括光照传感器、处理器和存储器;其中,处理器通过执行存储器中保存的VR全景图像生成程序实现前述实施例中公开的VR全景图像生成方法。Correspondingly, the present invention also discloses a VR panoramic image generating device, which is applied to a panoramic camera including N fisheye lenses, where N is an integer not less than 2; wherein, the device includes an illumination sensor, a processor, and a memory; wherein, The processor realizes the method for generating a VR panoramic image disclosed in the foregoing embodiments by executing the VR panoramic image generating program stored in the memory.

关于上述VR全景图像生成方法更加具体的过程可以参考前述实施例中公开的相应内容,在此不再进行赘述。Regarding the more specific process of the aforementioned method for generating a VR panoramic image, reference may be made to the corresponding content disclosed in the foregoing embodiments, and details are not repeated here.

进一步的,本发明还公开了一种全景相机,包括N个鱼眼镜头,其中,N为不小于2的整数,还包括前述实施例中公开的VR全景图像生成设备。关于该VR全景图像生成设备更加具体的构造可以参考前述实施例中公开的相应内容,在此不再进行赘述。Further, the present invention also discloses a panoramic camera, which includes N fisheye lenses, where N is an integer not less than 2, and also includes the VR panoramic image generation device disclosed in the foregoing embodiments. For a more specific structure of the VR panoramic image generating device, reference may be made to the corresponding contents disclosed in the foregoing embodiments, and details are not repeated here.

进一步的,本发明还公开了一种计算机可读存储介质,用于存储VR全景图像生成程序,其中,VR全景图像生成程序被处理器执行时实现前述实施例中公开的VR全景图像生成方法。关于上述VR全景图像生成方法更加具体的过程可以参考前述实施例中公开的相应内容,在此不再进行赘述。Further, the present invention also discloses a computer-readable storage medium for storing a VR panoramic image generation program, wherein the VR panoramic image generation program implements the VR panoramic image generation method disclosed in the foregoing embodiments when executed by a processor. Regarding the more specific process of the aforementioned method for generating a VR panoramic image, reference may be made to the corresponding content disclosed in the foregoing embodiments, and details are not repeated here.

本说明书中各个实施例采用递进的方式描述,每个实施例重点说明的都是与其它实施例的不同之处,各个实施例之间相同或相似部分互相参见即可。对于实施例公开的装置而言,由于其与实施例公开的方法相对应,所以描述的比较简单,相关之处参见方法部分说明即可。Each embodiment in this specification is described in a progressive manner, each embodiment focuses on the difference from other embodiments, and the same or similar parts of each embodiment can be referred to each other. As for the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and for the related information, please refer to the description of the method part.

专业人员还可以进一步意识到,结合本文中所公开的实施例描述的各示例的单元及算法步骤,能够以电子硬件、计算机软件或者二者的结合来实现,为了清楚地说明硬件和软件的可互换性,在上述说明中已经按照功能一般性地描述了各示例的组成及步骤。这些功能究竟以硬件还是软件方式来执行,取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能,但是这种实现不应认为超出本发明的范围。Professionals can further realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software or a combination of the two. In order to clearly illustrate the possible For interchangeability, in the above description, the composition and steps of each example have been generally described according to their functions. Whether these functions are executed by hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be regarded as exceeding the scope of the present invention.

结合本文中所公开的实施例描述的方法或算法的步骤可以直接用硬件、处理器执行的软件模块,或者二者的结合来实施。软件模块可以置于随机存储器(RAM)、内存、只读存储器(ROM)、电可编程ROM、电可擦除可编程ROM、寄存器、硬盘、可移动磁盘、CD-ROM、或技术领域内所公知的任意其它形式的存储介质中。The steps of the methods or algorithms described in connection with the embodiments disclosed herein may be directly implemented by hardware, software modules executed by a processor, or a combination of both. Software modules can be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other Any other known storage medium.

最后,还需要说明的是,在本文中,诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来,而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且,术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含,从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素,而且还包括没有明确列出的其他要素,或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下,由语句“包括一个……”限定的要素,并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。Finally, it should also be noted that in this text, relational terms such as first and second etc. are only used to distinguish one entity or operation from another, and do not necessarily require or imply that these entities or operations, any such actual relationship or order exists. Furthermore, the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus comprising a set of elements includes not only those elements, but also includes elements not expressly listed. other elements of or also include elements inherent in such a process, method, article, or device. Without further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising said element.

以上对本发明所提供的一种全景相机及其图像生成方法、系统、设备、存储介质进行了详细介绍,本文中应用了具体个例对本发明的原理及实施方式进行了阐述,以上实施例的说明只是用于帮助理解本发明的方法及其核心思想;同时,对于本领域的一般技术人员,依据本发明的思想,在具体实施方式及应用范围上均会有改变之处,综上所述,本说明书内容不应理解为对本发明的限制。A panoramic camera provided by the present invention and its image generation method, system, equipment, and storage medium have been introduced in detail above. In this paper, specific examples have been used to illustrate the principle and implementation of the present invention. The description of the above embodiments It is only used to help understand the method of the present invention and its core idea; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation and scope of application. In summary, The contents of this description should not be construed as limiting the present invention.

Claims (10)

1.一种VR全景图像生成方法,其特征在于,应用于包括N个鱼眼镜头的全景相机,N为不小于2的整数;其中,所述方法包括:1. A method for generating a VR panoramic image, characterized in that it is applied to a panoramic camera comprising N fisheye lenses, and N is an integer not less than 2; wherein the method comprises: 采集当前外界环境的实时光照强度;Collect the real-time light intensity of the current external environment; 判断实时光照强度是否小于预设光照强度阈值,如果是,则分别采集与每个鱼眼镜头对应的一组原始鱼眼图像,得到相应的N组原始鱼眼图像;其中,每组原始鱼眼图像中均包括多帧原始鱼眼图像;Determine whether the real-time light intensity is less than the preset light intensity threshold, and if so, collect a group of original fisheye images corresponding to each fisheye lens to obtain corresponding N groups of original fisheye images; wherein, each group of original fisheye The images all include multiple frames of original fisheye images; 分别对每组原始鱼眼图像进行融合处理,相应地得到N张融合图像;Perform fusion processing on each group of original fisheye images respectively, and obtain N fusion images accordingly; 对所述N张融合图像进行拼接处理,得到拼接后的全景图像。Stitching is performed on the N fusion images to obtain a stitched panoramic image. 2.根据权利要求1所述的VR全景图像生成方法,其特征在于,所述分别采集与每个鱼眼镜头对应的一组原始鱼眼图像,得到相应的N组原始鱼眼图像的步骤,包括:2. VR panorama image generation method according to claim 1, is characterized in that, described respectively collects one group of original fish-eye images corresponding to each fish-eye lens, obtains the step of corresponding N groups of original fish-eye images, include: 确定实时光照强度的强度等级,得到目标强度等级;Determine the intensity level of the real-time light intensity to obtain the target intensity level; 利用预设的强度等级与每组原始鱼眼图像的采集数量之间的映射关系,确定出与所述目标强度等级对应的采集数量,得到目标采集数量;Using the mapping relationship between the preset intensity level and the acquisition quantity of each group of original fisheye images, determine the acquisition quantity corresponding to the target intensity level, and obtain the target acquisition quantity; 根据所述目标采集数量,分别采集与每个鱼眼镜头对应的一组原始鱼眼图像,得到相应的N组原始鱼眼图像;其中,每组原始鱼眼图像的图像帧数与所述目标采集数量相等。According to the number of target acquisitions, a group of original fisheye images corresponding to each fisheye lens are respectively collected to obtain corresponding N groups of original fisheye images; wherein, the number of image frames of each group of original fisheye images is the same as that of the target The number of collections is equal. 3.根据权利要求1所述的VR全景图像生成方法,其特征在于,对每组原始鱼眼图像进行融合处理的过程,包括:3. VR panorama image generating method according to claim 1, is characterized in that, the process of fusion processing is carried out to each group of original fisheye images, comprising: 从该组原始鱼眼图像筛选出一帧原始鱼眼图像作为参考帧图像;A frame of original fish-eye image is selected from the group of original fish-eye images as a reference frame image; 将该组原始鱼眼图像中的其他原始鱼眼图像均与所述参考帧图像进行坐标对齐,得到相应的对齐后图像序列;All other original fisheye images in the group of original fisheye images are coordinate-aligned with the reference frame image to obtain a corresponding aligned image sequence; 对所述对齐后图像序列中的所有图像进行加权融合,得到相应的一张融合图像。All the images in the aligned image sequence are weighted and fused to obtain a corresponding fused image. 4.根据权利要求3所述的VR全景图像生成方法,其特征在于,所述从该组原始鱼眼图像筛选出一帧原始鱼眼图像作为参考帧图像的步骤,包括:4. VR panorama image generating method according to claim 3, is characterized in that, the described step of screening out a frame of original fish-eye image as reference frame image from this group of original fish-eye images, comprises: 分别计算该组原始鱼眼图像中每帧原始鱼眼图像的平均梯度,然后将平均梯度最大的一帧原始鱼眼图像作为参考帧图像。The average gradient of each frame of the original fisheye image in the group of original fisheye images is calculated respectively, and then the frame of the original fisheye image with the largest average gradient is used as the reference frame image. 5.根据权利要求3所述的VR全景图像生成方法,其特征在于,所述对所述对齐后图像序列中的所有图像进行加权融合,得到相应的一张融合图像的步骤,包括:5. The method for generating a VR panoramic image according to claim 3, wherein the step of carrying out weighted fusion to all images in the aligned image sequence to obtain a corresponding fusion image comprises: 分别根据所述对齐后图像序列中每一帧图像与所述参考帧图像所对应的图像坐标之间的像素灰度差值的绝对值,确定所述对齐后图像序列中每一帧图像的加权系数;Determine the weight of each frame of image in the aligned image sequence according to the absolute value of the pixel grayscale difference between each frame of image in the aligned image sequence and the image coordinates corresponding to the reference frame image coefficient; 根据所述对齐后图像序列中每一帧图像的加权系数,对所述对齐后图像序列中的所有图像进行加权融合,得到相应的一张融合图像。All the images in the aligned image sequence are weighted and fused according to the weight coefficient of each frame of image in the aligned image sequence to obtain a corresponding fused image. 6.根据权利要求1至5任一项所述的VR全景图像生成方法,其特征在于,所述对所述N张融合图像进行拼接处理,得到拼接后的全景图像的步骤,包括:6. The VR panoramic image generation method according to any one of claims 1 to 5, wherein the step of splicing the N fusion images to obtain the spliced panoramic image includes: 分别对每张融合图像进行平面展开,得到N张展开后图像;Carry out planar expansion on each fused image respectively to obtain N expanded images; 对所述N张展开后图像进行重合区域的配准处理,得到相应的配准结果;performing registration processing on overlapping regions of the N expanded images to obtain corresponding registration results; 根据所述配准结果,对所述N张展开后图像的重合区域进行融合处理,得到所述全景图像。According to the registration result, fusion processing is performed on overlapping regions of the N expanded images to obtain the panoramic image. 7.一种VR全景图像生成系统,其特征在于,应用于包括N个鱼眼镜头的全景相机,N为不小于2的整数;其中,所述系统包括:7. A VR panoramic image generation system, characterized in that it is applied to a panoramic camera comprising N fisheye lenses, where N is an integer not less than 2; wherein the system comprises: 光照强度采集模块,用于采集当前外界环境的实时光照强度;The light intensity acquisition module is used to collect the real-time light intensity of the current external environment; 光照强度判断模块,用于判断实时光照强度是否小于预设光照强度阈值;The light intensity judging module is used to judge whether the real-time light intensity is less than the preset light intensity threshold; 鱼眼图像采集模块,用于如果是,则分别采集与每个鱼眼镜头对应的一组原始鱼眼图像,得到相应的N组原始鱼眼图像;其中,每组原始鱼眼图像中均包括多帧原始鱼眼图像;The fisheye image acquisition module is used to collect a group of original fisheye images corresponding to each fisheye lens respectively to obtain corresponding N groups of original fisheye images; wherein, each group of original fisheye images includes Multi-frame original fisheye image; 鱼眼图像融合模块,用于分别对每组原始鱼眼图像进行融合处理,相应地得到N张融合图像;The fisheye image fusion module is used to fuse each group of original fisheye images respectively, and obtain N pieces of fusion images accordingly; 图像拼接模块,用于对所述N张融合图像进行拼接处理,得到拼接后的全景图像。An image splicing module, configured to splice the N fusion images to obtain a spliced panoramic image. 8.一种VR全景图像生成设备,其特征在于,应用于包括N个鱼眼镜头的全景相机,N为不小于2的整数;其中,所述设备包括光照传感器、处理器和存储器;其中,所述处理器通过执行所述存储器中保存的VR全景图像生成程序实现如权利要求1至6任一项所述的VR全景图像生成方法。8. A VR panoramic image generation device, characterized in that it is applied to a panoramic camera including N fisheye lenses, where N is an integer not less than 2; wherein the device includes an illumination sensor, a processor and a memory; wherein, The processor implements the VR panoramic image generation method according to any one of claims 1 to 6 by executing the VR panoramic image generation program stored in the memory. 9.一种全景相机,其特征在于,包括N个鱼眼镜头,其中,N为不小于2的整数,还包括如权利要求8所述的VR全景图像生成设备。9. A panoramic camera, characterized in that it comprises N fisheye lenses, wherein N is an integer not less than 2, and also comprises the VR panoramic image generating device as claimed in claim 8. 10.一种计算机可读存储介质,其特征在于,用于存储VR全景图像生成程序,其中,所述VR全景图像生成程序被处理器执行时实现如权利要求1至6任一项所述的VR全景图像生成方法。10. A computer-readable storage medium, characterized in that it is used to store a VR panoramic image generation program, wherein, when the VR panoramic image generation program is executed by a processor, the method according to any one of claims 1 to 6 is realized. A VR panorama image generation method.
CN201711237215.5A 2017-11-30 2017-11-30 A panoramic camera and its image generation method, system, device, and storage medium Pending CN107995421A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201711237215.5A CN107995421A (en) 2017-11-30 2017-11-30 A panoramic camera and its image generation method, system, device, and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201711237215.5A CN107995421A (en) 2017-11-30 2017-11-30 A panoramic camera and its image generation method, system, device, and storage medium

Publications (1)

Publication Number Publication Date
CN107995421A true CN107995421A (en) 2018-05-04

Family

ID=62034724

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201711237215.5A Pending CN107995421A (en) 2017-11-30 2017-11-30 A panoramic camera and its image generation method, system, device, and storage medium

Country Status (1)

Country Link
CN (1) CN107995421A (en)

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108765292A (en) * 2018-05-30 2018-11-06 中国人民解放军军事科学院国防科技创新研究院 Image split-joint method based on the fitting of space triangular dough sheet
CN108900766A (en) * 2018-06-15 2018-11-27 北京华捷艾米科技有限公司 A kind of panorama camera of the automatic enhancement device of panoramic picture and method and the application device
CN109379524A (en) * 2018-12-14 2019-02-22 上海杰图天下网络科技有限公司 Panoramic view data acquisition device based on industrial camera and industrial lens
CN111343385A (en) * 2020-03-23 2020-06-26 东软睿驰汽车技术(沈阳)有限公司 Method, device, equipment and storage medium for determining environment brightness
TWI704408B (en) * 2018-08-22 2020-09-11 宏碁股份有限公司 Omnidirectional camera apparatus and image mapping/combining method thereof
CN112651918A (en) * 2021-01-15 2021-04-13 北京小米松果电子有限公司 Image processing method and device, electronic device and storage medium
CN112818732A (en) * 2020-08-11 2021-05-18 腾讯科技(深圳)有限公司 Image processing method and device, computer equipment and storage medium
CN113567357A (en) * 2021-07-26 2021-10-29 杭州海康威视数字技术股份有限公司 Spectral data fusion method and device
CN116246340A (en) * 2022-12-29 2023-06-09 宁波极智云数据科技有限公司 Human action recognition method in complex environment
CN118413733A (en) * 2024-07-03 2024-07-30 圆周率科技(常州)有限公司 Panoramic image synthesis device, panoramic camera and image synthesis method

Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101361097A (en) * 2006-09-06 2009-02-04 卡西欧计算机株式会社 camera device
CN104601882A (en) * 2014-12-30 2015-05-06 广东欧珀移动通信有限公司 A panoramic shooting method and terminal
CN104835118A (en) * 2015-06-04 2015-08-12 浙江得图网络有限公司 Method for acquiring panorama image by using two fish-eye camera lenses
CN105635575A (en) * 2015-12-29 2016-06-01 宇龙计算机通信科技(深圳)有限公司 Imaging method, imaging device and terminal
CN105898159A (en) * 2016-05-31 2016-08-24 努比亚技术有限公司 Image processing method and terminal
CN106303283A (en) * 2016-08-15 2017-01-04 Tcl集团股份有限公司 A kind of panoramic image synthesis method based on fish-eye camera and system
CN106612397A (en) * 2016-11-25 2017-05-03 努比亚技术有限公司 Image processing method and terminal
US20170126972A1 (en) * 2015-10-30 2017-05-04 Essential Products, Inc. Imaging device and method for generating an undistorted wide view image
CN106683045A (en) * 2016-09-28 2017-05-17 深圳市优象计算技术有限公司 Binocular camera-based panoramic image splicing method
CN107169926A (en) * 2017-04-26 2017-09-15 腾讯科技(深圳)有限公司 Image processing method and device
CN107222680A (en) * 2017-06-30 2017-09-29 维沃移动通信有限公司 The image pickup method and mobile terminal of a kind of panoramic picture
CN107274346A (en) * 2017-06-23 2017-10-20 中国科学技术大学 Real-time panoramic video splicing system

Patent Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101361097A (en) * 2006-09-06 2009-02-04 卡西欧计算机株式会社 camera device
CN104601882A (en) * 2014-12-30 2015-05-06 广东欧珀移动通信有限公司 A panoramic shooting method and terminal
CN104835118A (en) * 2015-06-04 2015-08-12 浙江得图网络有限公司 Method for acquiring panorama image by using two fish-eye camera lenses
US20170126972A1 (en) * 2015-10-30 2017-05-04 Essential Products, Inc. Imaging device and method for generating an undistorted wide view image
CN105635575A (en) * 2015-12-29 2016-06-01 宇龙计算机通信科技(深圳)有限公司 Imaging method, imaging device and terminal
CN105898159A (en) * 2016-05-31 2016-08-24 努比亚技术有限公司 Image processing method and terminal
CN106303283A (en) * 2016-08-15 2017-01-04 Tcl集团股份有限公司 A kind of panoramic image synthesis method based on fish-eye camera and system
CN106683045A (en) * 2016-09-28 2017-05-17 深圳市优象计算技术有限公司 Binocular camera-based panoramic image splicing method
CN106612397A (en) * 2016-11-25 2017-05-03 努比亚技术有限公司 Image processing method and terminal
CN107169926A (en) * 2017-04-26 2017-09-15 腾讯科技(深圳)有限公司 Image processing method and device
CN107274346A (en) * 2017-06-23 2017-10-20 中国科学技术大学 Real-time panoramic video splicing system
CN107222680A (en) * 2017-06-30 2017-09-29 维沃移动通信有限公司 The image pickup method and mobile terminal of a kind of panoramic picture

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108765292A (en) * 2018-05-30 2018-11-06 中国人民解放军军事科学院国防科技创新研究院 Image split-joint method based on the fitting of space triangular dough sheet
CN108765292B (en) * 2018-05-30 2022-04-29 中国人民解放军军事科学院国防科技创新研究院 Image splicing method based on space triangular patch fitting
CN108900766A (en) * 2018-06-15 2018-11-27 北京华捷艾米科技有限公司 A kind of panorama camera of the automatic enhancement device of panoramic picture and method and the application device
TWI704408B (en) * 2018-08-22 2020-09-11 宏碁股份有限公司 Omnidirectional camera apparatus and image mapping/combining method thereof
CN109379524A (en) * 2018-12-14 2019-02-22 上海杰图天下网络科技有限公司 Panoramic view data acquisition device based on industrial camera and industrial lens
CN109379524B (en) * 2018-12-14 2024-06-04 上海杰图天下网络科技有限公司 Panoramic data acquisition device based on industrial camera and industrial lens
CN111343385A (en) * 2020-03-23 2020-06-26 东软睿驰汽车技术(沈阳)有限公司 Method, device, equipment and storage medium for determining environment brightness
CN112818732A (en) * 2020-08-11 2021-05-18 腾讯科技(深圳)有限公司 Image processing method and device, computer equipment and storage medium
CN112818732B (en) * 2020-08-11 2023-12-12 腾讯科技(深圳)有限公司 Image processing method, device, computer equipment and storage medium
CN112651918A (en) * 2021-01-15 2021-04-13 北京小米松果电子有限公司 Image processing method and device, electronic device and storage medium
CN112651918B (en) * 2021-01-15 2024-11-22 北京小米松果电子有限公司 Image processing method and device, electronic device, and storage medium
CN113567357A (en) * 2021-07-26 2021-10-29 杭州海康威视数字技术股份有限公司 Spectral data fusion method and device
CN113567357B (en) * 2021-07-26 2024-05-24 杭州海康威视数字技术股份有限公司 Fusion method and device of spectrum data
CN116246340A (en) * 2022-12-29 2023-06-09 宁波极智云数据科技有限公司 Human action recognition method in complex environment
CN118413733A (en) * 2024-07-03 2024-07-30 圆周率科技(常州)有限公司 Panoramic image synthesis device, panoramic camera and image synthesis method
CN118413733B (en) * 2024-07-03 2024-11-29 圆周率科技(常州)有限公司 Panoramic image synthesis device, panoramic camera and image synthesis method

Similar Documents

Publication Publication Date Title
CN107995421A (en) A panoramic camera and its image generation method, system, device, and storage medium
CN113382168B (en) Apparatus and method for storing overlapping regions of imaging data to produce an optimized stitched image
TWI712918B (en) Method, device and equipment for displaying images of augmented reality
CN109064397B (en) Image stitching method and system based on camera earphone
CN108600576B (en) Image processing apparatus, method and system, and computer-readable recording medium
JP6263623B2 (en) Image generation method and dual lens apparatus
CN104699842B (en) Picture display method and device
CN106550184B (en) Photo processing method and device
KR102229811B1 (en) Filming method and terminal for terminal
CN107948519A (en) Image processing method, device and equipment
CN106296589B (en) Panoramic image processing method and device
CN108564612A (en) Model display method, device, storage medium and electronic equipment
CN103945210A (en) Multi-camera photographing method for realizing shallow depth of field effect
US20140085422A1 (en) Image processing method and device
CN103985103A (en) Method and device for generating panoramic picture
CN111866523B (en) Panoramic video synthesis method and device, electronic equipment and computer storage medium
CN110278366B (en) A panoramic image blurring method, terminal and computer-readable storage medium
WO2013146041A1 (en) Image processing device and method, and imaging device
JP2015046044A (en) Image processing apparatus, image processing method, program, and imaging system
CN105791793A (en) Image processing method and electronic device thereof
CN106657809B (en) A kind of panorama 3D video-splicing system and method
CN113454980A (en) Panorama shooting method, electronic device and storage medium
EP3991132B1 (en) Imaging system, image processing apparatus, imaging device, and recording medium
CN108391116B (en) Whole body scanning device and method based on 3D imaging technology
CN106803920A (en) Image processing method and device and intelligent conference terminal

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
TA01 Transfer of patent application right

Effective date of registration: 20201012

Address after: 261031 north of Yuqing street, east of Dongming Road, high tech Zone, Weifang City, Shandong Province (Room 502, Geer electronic office building)

Applicant after: GoerTek Optical Technology Co.,Ltd.

Address before: 261205 Shandong province Weifang Comprehensive Bonded Zone Yuqing East Street to South High Tech two Road East Weifang Comprehensive Bonded Zone aide light industrial product processing base 1, 3, 5 workshop

Applicant before: Weifang Goertek Electronics Co.,Ltd.

TA01 Transfer of patent application right
RJ01 Rejection of invention patent application after publication

Application publication date: 20180504

RJ01 Rejection of invention patent application after publication