CN110099220B - Panoramic stitching method and device - Google Patents
Panoramic stitching method and device Download PDFInfo
- Publication number
- CN110099220B CN110099220B CN201910522276.9A CN201910522276A CN110099220B CN 110099220 B CN110099220 B CN 110099220B CN 201910522276 A CN201910522276 A CN 201910522276A CN 110099220 B CN110099220 B CN 110099220B
- Authority
- CN
- China
- Prior art keywords
- images
- panoramic
- original
- coordinates
- mapping
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 42
- 238000013507 mapping Methods 0.000 claims abstract description 116
- 230000003287 optical effect Effects 0.000 claims abstract description 76
- 238000012937 correction Methods 0.000 claims abstract description 51
- 238000012545 processing Methods 0.000 claims description 7
- 238000013135 deep learning Methods 0.000 claims description 5
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 230000003247 decreasing effect Effects 0.000 claims description 3
- 230000000694 effects Effects 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000007423 decrease Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000008094 contradictory effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/698—Control of cameras or camera modules for achieving an enlarged field of view, e.g. panoramic image capture
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/95—Computational photography systems, e.g. light-field imaging systems
- H04N23/957—Light-field or plenoptic cameras or camera modules
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computing Systems (AREA)
- Theoretical Computer Science (AREA)
- Studio Devices (AREA)
- Stereoscopic And Panoramic Photography (AREA)
Abstract
The invention provides a panoramic stitching method and a panoramic stitching device, wherein the panoramic stitching method comprises the following steps: acquiring a plurality of original images acquired by a plurality of lenses of a panoramic camera, wherein the optical centers of the plurality of lenses are different; switching a plurality of mapping tables with different optical center coincidence correction parameters according to different distances between a shooting scene of each original image in the plurality of original images and a lens for collecting the original image, wherein the plurality of mapping tables reflect the mapping relation between the coordinates of the plurality of original images and the coordinates of the plurality of panoramic spliced images, and the optical center coincidence correction parameters are used for correcting the position deviation of the optical centers of the plurality of lenses; converting a plurality of original images into a plurality of panoramic stitched images according to a plurality of mapping tables; the plurality of panoramic spliced images are spliced into the panoramic image, seamless splicing of the panoramic image can be realized, and the shooting distance for realizing seamless splicing in close-range shooting is shortened.
Description
Technical Field
The invention relates to the technical field of panoramic stitching, in particular to a panoramic stitching method and a panoramic stitching device.
Background
The panoramic camera can once only shoot 360 degrees panoramas, lets the user possess the scene that can't catch using traditional camera to the brand-new mode, except shooing, the panoramic camera can also be applied to the shooting of dynamic image, also is called panoramic camera. The range of the angle of view of the existing camera lenses is less than 360 degrees, so that the panoramic camera can shoot panoramic images only by being provided with at least two camera lenses. Different from the mode of rotating to shoot panoramic images of a traditional camera/smart phone, the panoramic camera can shoot images in the horizontal direction and the vertical direction simultaneously, and after a plurality of shot images are subjected to panoramic stitching, 360-degree panoramic images can be generated.
However, because the optical centers of the multiple lenses are often deviated in the actual carrying process, a plurality of images are easy to have obvious splicing seams at the splicing part, and particularly, splicing traces are more obvious when the shooting distance is small, so that the panoramic experience of a user is poor.
Disclosure of Invention
In view of this, embodiments of the present invention provide a panorama stitching method and apparatus, which can implement seamless stitching of a panorama image and shorten a shooting distance for implementing seamless stitching during close-range shooting.
According to a first aspect of the embodiments of the present invention, there is provided a panorama stitching method, including: acquiring a plurality of original images acquired by a plurality of lenses of a panoramic camera, wherein the optical centers of the plurality of lenses are different; switching a plurality of mapping tables with different optical center coincidence correction parameters according to different distances between a shooting scene of each original image in the plurality of original images and a lens for collecting the original image, wherein the plurality of mapping tables reflect the mapping relation between the coordinates of the plurality of original images and the coordinates of the plurality of panoramic spliced images, and the optical center coincidence correction parameters are used for correcting the position deviation of the optical centers of the plurality of lenses; converting a plurality of original images into a plurality of panoramic stitched images according to a plurality of mapping tables; and splicing the plurality of panoramic spliced images into a panoramic image.
In an embodiment of the present invention, the plurality of lenses includes a plurality of fisheye lenses, wherein before the acquiring of the plurality of original images captured by the plurality of lenses of the panoramic camera, the panoramic stitching method further includes: determining the mapping relation between the coordinates of a plurality of original fisheye images and the coordinates of a plurality of panoramic stitching images; and generating a plurality of mapping tables with different optical center coincidence correction parameters according to the mapping relation.
In an embodiment of the present invention, the determining a mapping relationship between coordinates of a plurality of original fisheye images and coordinates of a plurality of panoramic stitched images includes: determining a mapping relation between coordinates of the plurality of panoramic stitched images and coordinates of the spherical image; and determining the mapping relation between the coordinates of the spherical image and the coordinates of the plurality of original fisheye images.
In an embodiment of the present invention, the determining a mapping relationship between coordinates of the plurality of panoramic stitched images and coordinates of the spherical image includes: and determining the mapping relation between the coordinates of the plurality of panoramic stitched images and the coordinates of the spherical image in an equivalent rectangular projection mode.
In an embodiment of the present invention, the panorama stitching method further includes: and generating a panoramic video according to the plurality of panoramic images.
In an embodiment of the present invention, the switching between the plurality of mapping tables with different optical center coincidence correction parameters according to different distances between a shooting scene of each of the plurality of original images and a lens used for acquiring the original image includes: when the distance between a shooting scene of one original image of the original images and a lens is reduced, switching a mapping table with an increased optical center coincidence correction parameter; or switching the mapping table in which the optical center registration correction parameter decreases when the distance between the shot scene of one of the plurality of original images and the lens increases.
In an embodiment of the present invention, the switching between the plurality of mapping tables with different optical center coincidence correction parameters according to different distances between a shooting scene of each of the plurality of original images and a lens used for acquiring the original image includes: automatically identifying the distance between a shooting scene of an original image and a lens through deep learning, and switching a mapping table according to different distances; or the mapping table is manually switched by the operation of a user according to the difference of the distances.
In one embodiment of the invention, the optical center registration correction parameters include an x-axis variation component, a y-axis variation component, and a z-axis variation component.
According to a second aspect of the embodiments of the present invention, there is provided a panorama stitching apparatus, including: the system comprises an acquisition module, a processing module and a display module, wherein the acquisition module is used for acquiring a plurality of original images acquired by a plurality of lenses of the panoramic camera, and the optical centers of the lenses are different; the system comprises a switching module, a plurality of mapping tables and a plurality of panoramic splicing images, wherein the plurality of mapping tables are used for reflecting the mapping relation between the coordinates of the plurality of original images and the coordinates of the plurality of panoramic splicing images, and the optical center superposition correction parameters are used for correcting the position deviation of the optical centers of the plurality of lenses; the conversion module is used for converting the plurality of original images into a plurality of panoramic mosaic images according to the plurality of mapping tables; and the splicing module is used for splicing the plurality of panoramic spliced images into panoramic images.
According to a third aspect of embodiments of the present invention, there is provided a computer-readable storage medium having stored thereon computer-executable instructions that, when executed by a processor, implement the panorama stitching method as described above.
According to the technical scheme provided by the embodiment of the invention, a plurality of original images collected by a plurality of lenses of a panoramic camera are acquired; switching a plurality of mapping tables with different optical center coincidence correction parameters according to different distances between a shooting scene of each original image in the plurality of original images and a lens for acquiring the image; converting a plurality of original images into a plurality of panoramic stitched images according to a plurality of mapping tables; the plurality of panoramic spliced images are spliced into the panoramic image, the images can be spliced by using different mapping tables aiming at scenes such as a long shot scene, a middle shot scene, a short shot scene and the like, a better splicing effect is generated, and the shooting distance of seamless splicing can be shortened during short-shot shooting.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic flow chart of a panoramic stitching method according to an embodiment of the present invention.
Fig. 2 is a schematic flow chart of a panoramic stitching method according to another embodiment of the present invention.
Fig. 3 is a block diagram of a panorama stitching apparatus according to an embodiment of the present invention.
Fig. 4 is a block diagram of a panorama stitching apparatus according to another embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The panorama stitching method provided by the embodiment of the invention can be executed by computer equipment (such as a server), namely, the materials shot by each lens are stitched and fused in the later period; the stitching may also be performed by a panoramic camera, and the stitching may be performed in real time, which is not limited in the present invention.
Fig. 1 is a schematic flow chart of a panoramic stitching method according to an embodiment of the present invention. As shown in fig. 1, the panorama stitching method includes the following steps.
S110: a plurality of original images collected by a plurality of lenses of the panoramic camera are acquired, and optical centers of the plurality of lenses are different.
The lens may be a fisheye lens, a spherical lens, or a general lens, which is not limited in the present invention. The plurality of lenses of the panoramic camera may be two or three lenses, and the number of lenses is not particularly limited in the present invention. For example, a panoramic camera is composed of two back-to-back fisheye lenses, each fisheye lens has a field angle exceeding 180 degrees, and the two fisheye lenses respectively capture fisheye images with sufficiently large field angles at the same time.
S120: and switching a plurality of mapping tables with different optical center coincidence correction parameters according to different distances between the shooting scene of each original image in the plurality of original images and a lens for acquiring the original image.
Specifically, a mapping table for generating a mapping relationship between coordinates of an original image shot by a lens and coordinates of a panoramic stitched image generated by converting the coordinates of the original image shot by the lens may be stored, wherein a plurality of sets of mapping tables with different optical center registration correction parameters are generated according to the distance between the shot by the lens, for example, the mapping table records that the coordinates on each panoramic image correspond to the camera number (which lens) of the original fisheye image and the corresponding pixel coordinates of the original fisheye image, and for example, three decimal places may be reserved. In addition, reading the pixels of the coordinates corresponding to the fisheye image according to the mapping table, and if the pixels are decimal, performing interpolation to obtain a final panoramic image. The optical center coincidence correction parameter is used for correcting the positional deviation of the optical centers of the plurality of lenses, and includes an x-axis variation component (Δ x), a y-axis variation component (Δ y), and a z-axis variation component (Δ z), and when the positional deviation of the optical centers of the two lenses is large, the Δ x, Δ y, and Δ z components are large. The optical center of the current lens is moved to be superposed with the optical center coordinate of the other lens by increasing the components of delta x, delta y and delta z to the optical center coordinate of the current lens, so that the two lenses are concentric to achieve a better splicing effect.
The farther a shooting scene is, the smaller the optical center coincidence correction parameter is; the closer the shooting scene is, the larger the optical center coincidence correction parameter is. When a long shot is shot, the influence of the position deviation of the optical centers of the plurality of lenses is small, the plurality of lenses can be approximate to be concentric, and the increased components of delta x, delta y and delta z of the optical center coordinates can be zero; when a close shot is shot, the positional deviation of the optical centers of the plurality of lenses has a large influence, and Δ x, Δ y, and Δ z components of the optical center coordinates increase are large.
S130: and converting the plurality of original images into a plurality of panoramic stitched images according to the plurality of mapping tables.
And selecting mapping tables with different optical center coincidence correction parameters according to the distance of a shooting scene, and converting images shot by each lens into panoramic mosaic images to be spliced.
S140: and splicing the plurality of panoramic spliced images into a panoramic image.
And splicing the plurality of panoramic spliced images to be spliced into the panoramic image by means of a splicing fusion algorithm and the like, wherein the specific splicing manner is not limited by the invention. The panoramic image may be 360 degrees, or a large-field-angle panoramic image spliced according to the user requirement, which is not limited in the present invention.
According to the technical scheme provided by the embodiment of the invention, a plurality of original images collected by a plurality of lenses of a panoramic camera are acquired; switching a plurality of mapping tables with different optical center coincidence correction parameters according to different distances between a shooting scene of each original image in the plurality of original images and a lens for acquiring the image; converting a plurality of original images into a plurality of panoramic stitched images according to a plurality of mapping tables; the plurality of panoramic spliced images are spliced into the panoramic image, the images can be spliced by using different mapping tables aiming at scenes such as a long shot scene, a middle shot scene, a short shot scene and the like, a better splicing effect is generated, and the shooting distance of seamless splicing can be shortened during short-shot shooting.
In another embodiment of the present invention, the plurality of lenses comprises a plurality of fisheye lenses, wherein before the acquiring the plurality of original images captured by the plurality of lenses of the panoramic camera, the panoramic stitching method further comprises: determining the mapping relation between the coordinates of a plurality of original fisheye images and the coordinates of a plurality of panoramic stitching images; and generating a plurality of mapping tables with different optical center coincidence correction parameters according to the mapping relation.
In another embodiment of the present invention, the determining a mapping relationship between coordinates of a plurality of original fisheye images and coordinates of a plurality of panoramic stitched images includes: determining a mapping relation between coordinates of the plurality of panoramic stitched images and coordinates of the spherical image; and determining the mapping relation between the coordinates of the spherical image and the coordinates of the plurality of original fisheye images.
Specifically, the fisheye image is mapped to the same spherical surface through the internal participation of the camera, wherein the internal participation comprises a focal length (or a field of view), and the external participation comprises a pitch angle (pitch), a yaw angle (yaw) and a roll angle (roll). The invention further adds external reference light center coincidence correction parameters delta x, delta y and delta z. Mapping the coordinates of the panoramic mosaic images to spherical coordinates to obtain x, y and z component coordinates, and adding delta x, delta y and delta z components to the x, y and z component coordinates to obtain new spherical x ', y ' and z ' component coordinates; performing spherical coordinate transformation according to a pitch angle (pitch), a yaw angle (yaw) and a roll angle (roll); and according to the mapping relation between the spherical image coordinates and the fisheye image coordinates, obtaining fisheye coordinates x ' and y ' from the x ', y ' and z ' component coordinates. And according to the point coordinates of each point coordinate of the panoramic mosaic image corresponding to the point coordinates of the original fisheye image, the conversion from the fisheye image to the panoramic mosaic image can be realized. In addition, in the process of optimizing the parameters according to the control points, the average distance of the control points can be reduced from about 2 pixels to about 1 pixel by increasing the components of Δ x, Δ y and Δ z.
In another embodiment of the present invention, the determining the mapping relationship between the coordinates of the plurality of panoramic stitched images and the coordinates of the spherical image includes: and determining the mapping relation between the coordinates of the plurality of panoramic stitched images and the coordinates of the spherical image in an equivalent rectangular projection mode.
Specifically, the fisheye image may be converted into an equirectangular panoramic image by an equirectangular projection (equirectangular).
In an embodiment of the present invention, the panorama stitching method further includes: and generating a panoramic video according to the plurality of panoramic images.
Specifically, each frame of spliced panoramic image is arranged according to a time sequence, and each arranged panoramic image frame forms a panoramic video.
In another embodiment of the present invention, the switching between the plurality of mapping tables with different optical center registration correction parameters according to the difference between the shooting scene of each of the plurality of original images and the lens used for acquiring the original image includes: when the distance between a shooting scene of one original image of the original images and a lens is reduced, switching a mapping table with an increased optical center coincidence correction parameter; or switching the mapping table in which the optical center registration correction parameter decreases when the distance between the shot scene of one of the plurality of original images and the lens increases.
For example, when a close-up image is processed, the mapping tables in which Δ x, Δ y, and Δ z components are large are switched; when processing a distant view image, the mapping tables having smaller Δ x, Δ y, and Δ z components are switched. The relationship between the optical center coincidence correction parameters (Δ x, Δ y, and Δ z components) and the distance between the shooting scene and the lens may be an empirical value, which is not limited by the present invention.
In another embodiment of the present invention, the switching between the plurality of mapping tables with different optical center registration correction parameters according to the difference between the shooting scene of each of the plurality of original images and the lens used for acquiring the original image includes: automatically identifying the distance between a shooting scene of an original image and a lens through deep learning, and switching a mapping table according to different distances; or the mapping table is manually switched by the operation of a user according to the difference of the distances.
Through deep learning, the shooting scene can be automatically identified to be a short scene, a medium scene, a long scene or the like, and the mapping tables with different components of delta x, delta y and delta z can be automatically switched according to the identified distance result. The user may also judge the distance of the shooting scene by himself, and manually switch the mapping table, etc., which is not limited in the present invention.
Fig. 2 is a schematic flow chart of a panoramic stitching method according to another embodiment of the present invention. As shown in fig. 2, the panorama stitching method includes the following steps.
S210: and determining the mapping relation between the coordinates of the plurality of original fisheye images and the coordinates of the plurality of panoramic stitching images.
S220: and generating a plurality of mapping tables with different optical center coincidence correction parameters according to the mapping relation.
S230: and acquiring a plurality of original fisheye images collected by a plurality of fisheye lenses of the panoramic camera.
S240: and automatically identifying the distance between the shooting scene of each original fisheye image and the fisheye lens.
S250: when the distance is longer, switching a mapping table with smaller optical center coincidence correction parameters; and when the distance is shorter, switching the mapping table with larger optical center coincidence correction parameters.
S260: and converting the plurality of original fisheye images into a plurality of panoramic stitching images according to the plurality of switched mapping tables.
S270: and splicing the plurality of panoramic spliced images into a panoramic image.
S280: and generating a panoramic video according to the plurality of panoramic images.
According to the technical scheme of the embodiment of the invention, a plurality of mapping tables are generated and stored through the mapping relation between the coordinates of the fisheye images and the coordinates of the panoramic stitching images, when the plurality of fisheye images are obtained, the plurality of mapping tables with different optical center coincidence correction parameters are switched according to the distance between the shooting scene of the fisheye images and a fisheye lens, the plurality of fisheye images are converted into the plurality of panoramic stitching images according to the plurality of mapping tables, and the panoramic stitching images are stitched into the panoramic images, so that the image stitching can be carried out by using different mapping tables aiming at the scenes such as long shot scenes, middle shot scenes, short shot scenes and the like, a better stitching effect is produced, the shooting distance for realizing seamless stitching during close shot can be shortened from 2-4 meters to 0.5 meter, in addition, the panoramic video with better stitching effect can be produced according to the plurality of panoramic images, and the experience of the panoramic video is.
All the above-mentioned optional technical solutions can be combined arbitrarily to form the optional embodiments of the present invention, and are not described herein again.
The following are embodiments of the apparatus of the present invention that may be used to perform embodiments of the method of the present invention. For details which are not disclosed in the embodiments of the apparatus of the present invention, reference is made to the embodiments of the method of the present invention.
Fig. 3 is a block diagram of a panorama stitching apparatus according to an embodiment of the present invention. As shown in fig. 3, the panorama stitching apparatus 300 includes:
an obtaining module 310, configured to obtain a plurality of original images collected by a plurality of lenses of a panoramic camera, where optical centers of the plurality of lenses are different;
the switching module 320 is configured to switch multiple mapping tables with different optical center coincidence correction parameters according to different distances between a shooting scene of each of the multiple original images and a lens used for acquiring the original image, where the multiple mapping tables reflect mapping relationships between coordinates of the multiple original images and coordinates of the multiple panoramic stitched images, and the optical center coincidence correction parameters are used for correcting position deviations of optical centers of the multiple lenses;
the conversion module 330 is configured to convert the plurality of original images into a plurality of panoramic stitched images according to the plurality of mapping tables;
and the stitching module 340 is configured to stitch the plurality of panoramic stitched images into a panoramic image.
According to the technical scheme provided by the embodiment of the invention, a plurality of original images collected by a plurality of lenses of a panoramic camera are acquired; switching a plurality of mapping tables with different optical center coincidence correction parameters according to different distances between a shooting scene of each original image in the plurality of original images and a lens for acquiring the image; converting a plurality of original images into a plurality of panoramic stitched images according to a plurality of mapping tables; the plurality of panoramic spliced images are spliced into the panoramic image, the images can be spliced by using different mapping tables aiming at scenes such as a long shot scene, a middle shot scene, a short shot scene and the like, a better splicing effect is generated, and the shooting distance of seamless splicing can be shortened during short-shot shooting.
In another embodiment of the present invention, the plurality of lenses includes a plurality of fisheye lenses, and the panorama stitching apparatus further includes a determining module 350, configured to determine a mapping relationship between coordinates of the plurality of original fisheye images and coordinates of the plurality of panorama stitching images before the obtaining of the plurality of original images captured by the plurality of lenses of the panorama camera; and a generating module 360, configured to generate a plurality of mapping tables with different optical center coincidence correction parameters according to the mapping relationship.
In another embodiment of the present invention, the determining module 350 is further configured to determine a mapping relationship between coordinates of the plurality of panoramic stitched images and coordinates of the spherical image; and determining the mapping relation between the coordinates of the spherical image and the coordinates of the plurality of original fisheye images.
In another embodiment of the present invention, the determining module 350 is further configured to determine a mapping relationship between coordinates of the plurality of panoramic stitched images and coordinates of the spherical image through an equivalent rectangular projection manner.
In another embodiment of the present invention, the generating module 360 is further configured to generate a panoramic video from the plurality of panoramic images.
In another embodiment of the present invention, the switching module 320 is further configured to switch the mapping table in which the optical center coincidence correction parameter is increased when the distance between the shot scene of one of the plurality of original images and the lens is decreased; or switching the mapping table in which the optical center registration correction parameter decreases when the distance between the shot scene of one of the plurality of original images and the lens increases.
In another embodiment of the present invention, the switching module 320 is further configured to automatically identify a distance between a shooting scene of the original image and a shot through deep learning, and switch the mapping table according to different distances; or the mapping table is manually switched by the operation of a user according to the difference of the distances.
In another embodiment of the present invention, the optical center registration correction parameters include an x-axis variation component, a y-axis variation component, and a z-axis variation component.
The implementation process of the functions and actions of each module in the above device is specifically described in the implementation process of the corresponding step in the above method, and is not described herein again.
Fig. 4 is a block diagram of a panorama stitching apparatus 400 according to another embodiment of the present invention.
Referring to fig. 4, apparatus 400 includes a processing component 410 that further includes one or more processors and memory resources, represented by memory 420, for storing instructions, such as applications, that are executable by processing component 410. The application programs stored in memory 420 may include one or more modules that each correspond to a set of instructions. Further, the processing component 410 is configured to execute instructions to perform the panorama stitching method described above.
The apparatus 400 may also include a power supply component configured to perform power management of the apparatus 400, a wired or wireless network interface configured to connect the apparatus 400 to a network, and an input output (I/O) interface. The apparatus 400 may operate based on an operating system, such as Windows Server, stored in the memory 420TM，Mac OS XTM，UnixTM，LinuxTM，FreeBSDTMOr the like.
A non-transitory computer readable storage medium having instructions stored thereon which, when executed by a processor of the apparatus 400, enable the apparatus 400 to perform a panorama stitching method, comprising: acquiring a plurality of original images acquired by a plurality of lenses of a panoramic camera, wherein the optical centers of the plurality of lenses are different; switching a plurality of mapping tables with different optical center coincidence correction parameters according to different distances between a shooting scene of each original image in the plurality of original images and a lens for acquiring the images, wherein the plurality of mapping tables reflect the mapping relation between the coordinates of the plurality of original images and the coordinates of the plurality of panoramic mosaic images, and the optical center coincidence correction parameters are used for correcting the position deviation of the optical centers of the plurality of lenses; converting a plurality of original images into a plurality of panoramic stitched images according to a plurality of mapping tables; and splicing the plurality of panoramic spliced images into a panoramic image.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention or a part thereof, which essentially contributes to the prior art, can be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program check codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
It should be noted that the combination of the features in the present application is not limited to the combination described in the claims or the combination described in the embodiments, and all the features described in the present application may be freely combined or combined in any manner unless contradictory to each other.
It should be noted that the above-mentioned embodiments are only specific examples of the present invention, and obviously, the present invention is not limited to the above-mentioned embodiments, and many similar variations exist. All modifications which would occur to one skilled in the art and which are, therefore, directly derived or suggested from the disclosure herein are deemed to be within the scope of the present invention.
It should be understood that the terms such as first, second, etc. used in the embodiments of the present invention are only used for clearly describing the technical solutions of the embodiments of the present invention, and are not used to limit the protection scope of the present invention.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (9)
1. A panoramic stitching method is characterized by comprising the following steps:
acquiring a plurality of original images acquired by a plurality of lenses of a panoramic camera, wherein the optical centers of the plurality of lenses are different;
switching a plurality of mapping tables with different optical center coincidence correction parameters according to different distances between a shooting scene of each original image in the plurality of original images and a lens for collecting the original image, wherein the plurality of mapping tables reflect the mapping relation between the coordinates of the plurality of original images and the coordinates of the plurality of panoramic stitched images, and the optical center coincidence correction parameters are used for correcting the position deviation of the optical centers of the plurality of lenses;
converting the plurality of original images into the plurality of panoramic stitched images according to the plurality of mapping tables;
stitching the plurality of panoramic stitched images into a panoramic image,
wherein, according to the difference between the shooting scene of each original image in the plurality of original images and the distance of the lens for collecting the original image, switching a plurality of mapping tables with different optical center coincidence correction parameters, comprises:
when the distance between the shooting scene of one original image of the original images and the lens is reduced, switching the mapping table with the increased optical center coincidence correction parameter; or
And when the distance between the shot scene of one original image of the plurality of original images and the lens is increased, switching the mapping table with the decreased optical center coincidence correction parameter.
2. A panorama stitching method as recited in claim 1, wherein the plurality of lenses comprises a plurality of fisheye lenses, wherein prior to the obtaining a plurality of raw images captured by the plurality of lenses of the panorama camera, the method further comprises:
determining the mapping relation between the coordinates of a plurality of original fisheye images and the coordinates of a plurality of panoramic stitching images;
and generating the plurality of mapping tables with different optical center coincidence correction parameters according to the mapping relation.
3. The panorama stitching method of claim 2, wherein determining a mapping relationship of coordinates of the plurality of original fisheye images and coordinates of the plurality of panorama stitched images comprises:
determining a mapping relation between the coordinates of the panoramic stitching images and the coordinates of the spherical images; and
and determining the mapping relation between the coordinates of the spherical image and the coordinates of the plurality of original fisheye images.
4. The panorama stitching method of claim 3, wherein the determining a mapping relationship of coordinates of the plurality of panorama stitched images to coordinates of spherical images comprises:
and determining the mapping relation between the coordinates of the plurality of panoramic stitched images and the coordinates of the spherical image in an equivalent rectangular projection mode.
5. The panorama stitching method of claim 1, further comprising:
and generating a panoramic video according to the plurality of panoramic images.
6. The panorama stitching method of claim 1, wherein switching the plurality of mapping tables with different optical center coincidence correction parameters according to a difference between a shot scene of each of the plurality of original images and a lens from which the original image was captured comprises:
automatically identifying the distance between the shooting scene of the original image and the lens through deep learning, and switching a mapping table according to different distances; or
And manually switching the mapping table by the user operation according to the difference of the distances.
7. The panorama stitching method of any one of claims 1-6, wherein the optical center registration correction parameters include an x-axis variation component, a y-axis variation component, and a z-axis variation component.
8. A panorama splicing apparatus, comprising:
the system comprises an acquisition module, a processing module and a display module, wherein the acquisition module is used for acquiring a plurality of original images acquired by a plurality of lenses of a panoramic camera, and the optical centers of the lenses are different;
the switching module is used for switching a plurality of mapping tables with different optical center coincidence correction parameters according to different distances between a shooting scene of each original image in the plurality of original images and a lens for collecting the original image, wherein the plurality of mapping tables reflect the mapping relation between the coordinates of the plurality of original images and the coordinates of the plurality of panoramic mosaic images, and the optical center coincidence correction parameters are used for correcting the position deviation of the optical centers of the plurality of lenses;
the conversion module is used for converting the plurality of original images into the plurality of panoramic stitched images according to the plurality of mapping tables;
a stitching module for stitching the plurality of panoramic stitched images into a panoramic image,
wherein the switching module is further configured to:
when the distance between the shooting scene of one original image of the original images and the lens is reduced, switching the mapping table with the increased optical center coincidence correction parameter; or
And when the distance between the shot scene of one original image of the plurality of original images and the lens is increased, switching the mapping table with the decreased optical center coincidence correction parameter.
9. A computer-readable storage medium having computer-executable instructions stored thereon, which when executed by a processor implement the panorama stitching method of any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910522276.9A CN110099220B (en) | 2019-06-17 | 2019-06-17 | Panoramic stitching method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910522276.9A CN110099220B (en) | 2019-06-17 | 2019-06-17 | Panoramic stitching method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110099220A CN110099220A (en) | 2019-08-06 |
| CN110099220B true CN110099220B (en) | 2021-04-13 |
Family
ID=67450997
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910522276.9A Active CN110099220B (en) | 2019-06-17 | 2019-06-17 | Panoramic stitching method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110099220B (en) |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111193920B (en) * | 2019-12-31 | 2020-12-18 | 重庆特斯联智慧科技股份有限公司 | A method and system for stereoscopic splicing of video images based on deep learning network |
| CN112669199B (en) * | 2020-12-16 | 2022-06-21 | 影石创新科技股份有限公司 | Image stitching method, computer-readable storage medium and computer device |
| CN112714260A (en) * | 2021-01-09 | 2021-04-27 | 深圳市二郎神视觉科技有限公司 | Panoramic photographing system and photographing method |
| WO2022246608A1 (en) * | 2021-05-24 | 2022-12-01 | 深圳市大疆创新科技有限公司 | Method for generating panoramic video, apparatus, and mobile platform |
| CN113870107B (en) * | 2021-08-18 | 2025-08-08 | 广州方硅信息技术有限公司 | Panoramic image stitching method, stitching device, stitching equipment and storage medium |
| CN113781373B (en) * | 2021-08-26 | 2024-08-23 | 云从科技集团股份有限公司 | Image fusion method, device and computer storage medium |
| CN115835022A (en) * | 2022-12-21 | 2023-03-21 | 苏州科达科技股份有限公司 | Panoramic stitching method, device, electronic equipment and storage medium |
| CN117593225B (en) * | 2023-11-14 | 2024-05-28 | 自行科技(武汉)有限公司 | Optical center deviation processing method, system, device and storage medium for electronic rearview mirror |
| CN118071655B (en) * | 2024-04-18 | 2024-08-30 | 松立控股集团股份有限公司 | Fisheye image correction and stitching method and system |
| CN118118645B (en) * | 2024-04-23 | 2024-07-19 | 北京工业大学 | Panoramic farm implementation method and device based on VR technology |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101521745A (en) * | 2009-04-14 | 2009-09-02 | 王广生 | Multi-lens optical center superposing type omnibearing shooting device and panoramic shooting and retransmitting method |
| CN102143305A (en) * | 2010-02-02 | 2011-08-03 | 华为终端有限公司 | Image pickup method and system |
| CN105894451A (en) * | 2016-03-30 | 2016-08-24 | 沈阳泰科易科技有限公司 | Method and device for splicing panoramic image |
| CN108171759A (en) * | 2018-01-26 | 2018-06-15 | 上海小蚁科技有限公司 | The scaling method of double fish eye lens panorama cameras and device, storage medium, terminal |
| CN108200360A (en) * | 2018-01-12 | 2018-06-22 | 深圳市粒视界科技有限公司 | A kind of real-time video joining method of more fish eye lens panoramic cameras |
| CN109859261A (en) * | 2019-01-07 | 2019-06-07 | 宁波舜宇光电信息有限公司 | A kind of quality determining method of fish-eye optical center localization method and device and camera module |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101581569B (en) * | 2009-06-17 | 2011-01-12 | 北京信息科技大学 | Calibration Method of Structural Parameters of Binocular Vision Sensing System |
| CN103177432B (en) * | 2013-03-28 | 2015-11-18 | 北京理工大学 | A kind of by coded aperture camera acquisition panorama sketch method |
| US9363449B1 (en) * | 2014-11-13 | 2016-06-07 | Futurewei Technologies, Inc. | Parallax tolerant video stitching with spatial-temporal localized warping and seam finding |
| CN107424118A (en) * | 2017-03-28 | 2017-12-01 | 天津大学 | Based on the spherical panorama mosaic method for improving Lens Distortion Correction |
-
2019
- 2019-06-17 CN CN201910522276.9A patent/CN110099220B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101521745A (en) * | 2009-04-14 | 2009-09-02 | 王广生 | Multi-lens optical center superposing type omnibearing shooting device and panoramic shooting and retransmitting method |
| CN102143305A (en) * | 2010-02-02 | 2011-08-03 | 华为终端有限公司 | Image pickup method and system |
| CN105894451A (en) * | 2016-03-30 | 2016-08-24 | 沈阳泰科易科技有限公司 | Method and device for splicing panoramic image |
| CN108200360A (en) * | 2018-01-12 | 2018-06-22 | 深圳市粒视界科技有限公司 | A kind of real-time video joining method of more fish eye lens panoramic cameras |
| CN108171759A (en) * | 2018-01-26 | 2018-06-15 | 上海小蚁科技有限公司 | The scaling method of double fish eye lens panorama cameras and device, storage medium, terminal |
| CN109859261A (en) * | 2019-01-07 | 2019-06-07 | 宁波舜宇光电信息有限公司 | A kind of quality determining method of fish-eye optical center localization method and device and camera module |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110099220A (en) | 2019-08-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110099220B (en) | Panoramic stitching method and device | |
| CN112311965B (en) | Virtual shooting method, device, system and storage medium | |
| CN109003311B (en) | Calibration method of fisheye lens | |
| JP6263623B2 (en) | Image generation method and dual lens apparatus | |
| CN106934777B (en) | Scanning image acquisition method and device | |
| US9892488B1 (en) | Multi-camera frame stitching | |
| CN111385461B (en) | Panoramic shooting method and device, camera and mobile terminal | |
| US10063792B1 (en) | Formatting stitched panoramic frames for transmission | |
| CN105530431A (en) | Reflective panoramic imaging system and method | |
| WO2021022989A1 (en) | Calibration parameter obtaining method and apparatus, processor, and electronic device | |
| CN114697623A (en) | Projection surface selection and projection image correction method and device, projector and medium | |
| CN116433769B (en) | Space calibration method, device, electronic equipment and storage medium | |
| CN116416701A (en) | Inspection method, inspection device, electronic equipment and storage medium | |
| CN111866523B (en) | Panoramic video synthesis method and device, electronic equipment and computer storage medium | |
| CN114640833A (en) | Projection picture adjusting method and device, electronic equipment and storage medium | |
| CN112995491A (en) | Video generation method and device, electronic equipment and computer storage medium | |
| CN111105351B (en) | Video sequence image splicing method and device | |
| CN117765174A (en) | Three-dimensional reconstruction method, device and equipment based on monocular cradle head camera | |
| CN112312041A (en) | Image correction method and device based on shooting, electronic equipment and storage medium | |
| CN104184936A (en) | Image focusing processing method and system based on light field camera | |
| CN111353945B (en) | Fisheye image correction method, device and storage medium | |
| JP2017103695A (en) | Image processing apparatus, image processing method, and program thereof | |
| CN106791358B (en) | Terminal photographing method and device and terminal | |
| KR20250044193A (en) | 3D drawing shooting method and 3D shooting system | |
| CN115775205A (en) | Multi-camera picture stitching method, device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |