CN108876712B - Spherical closed cabin panoramic display method based on double projection transformation - Google Patents

Abstract

The invention discloses a method for panoramic display of spherical enclosed cockpit based on double projection transformation. , spherical display, including: spherical segmentation: obtaining N equal-sized polygon images; block storage: storing the polygon images in the buffer in blocks; chain addressing: in each polygon image, the pixels are chained according to the chain. Sequence addressing and recoding; chain scan: scan and drive the physical screen in a chain sequence; block display: read the storage content of the polygon image in the buffer, and display it on the physical screen corresponding to this area. This method proposes a new addressing scanning method suitable for any special-shaped screen, suitable for any splicing spherical display, suitable for spherical panoramic display occasions in a closed environment, and can be used for various closed cockpits or simulated training cabins. for a better immersive experience.

Description

Panoramic display method of spherical closed cockpit based on double projection transformation

technical field

The invention belongs to the technical field of panoramic display, and in particular relates to a method for panoramic display of a spherical closed cockpit based on double projection transformation.

Background technique

At present, the traditional cockpit display system mostly adopts the multi-unit display method, that is, multiple sets of screens are spliced together to form a large-screen display effect. This method requires a multi-screen graphics card to process the original image, and output the split image to a multi-channel display for display. The display area of this method can only be the front-view area of the viewer, and it is difficult to generate a 360-degree panoramic display by splicing, and it cannot bring a superior immersive experience in a closed environment.

The panoramic display system based on virtual reality is a relatively popular panoramic display method in the existing solutions, but the user must wear VR glasses, which is very inconvenient, and the method also has the problem of a narrow display viewing angle. If there is a delay in the display, it is easy to bring a bad experience of dizziness to the user.

Other more mature panoramic display technologies are projection panoramic displays, such as the CAVE immersive projection display system. This is a large-scale VR display system with good immersion and good interactive means, but its display principle is complex and requires a high-end graphics processing system to cooperate with it, which is often expensive, cannot be widely promoted, and is not suitable for Enclosed cockpit display. Projection displays have better experience in large-scale applications such as wrap-around cylindrical or dome screens, which can provide a wider field of view and a better immersive experience. However, in the cockpit display system, due to the narrow space, the projection display method is easily blocked by the movement of people, which affects the viewing effect.

SUMMARY OF THE INVENTION

The technical problem to be solved by the present invention is:

In order to solve the technical deficiencies of the existing closed cockpit panoramic display system such as difficult implementation and weak immersive experience, the present invention provides a spherical closed cockpit panoramic display method based on double projection transformation.

The present invention adopts the following technical solutions for solving the above-mentioned technical problems:

The present invention provides a method for displaying panorama of spherical enclosed cockpit based on double projection transformation, and the steps include:

S01, image acquisition: acquire at least M groups of images;

S02, panoramic stitching: the M groups of images are subjected to projection transformation to obtain a complete spherical panoramic image; the projection transformation includes cylindrical projection transformation and spherical fitting projection transformation;

S03, spherical display, specifically including:

Spherical segmentation: segment the spherical panoramic image to obtain N equal-sized polygon images;

Block storage: store the polygon image in the buffer in blocks;

Chain addressing: address the pixels in a chain sequence within each polygon image and re-encode;

Chain Scan: Scan and drive the physical screen in a chain sequence;

Block display: read the storage content of the polygon image in the buffer, and display it on the physical screen corresponding to this area.

A kind of spherical closed cockpit panorama display method based on bi-projection transformation as mentioned above, further, S01. Image collection said M groups of images are collected by M cameras respectively, and the cameras form an average β angle with each other distributed on a circular structure.

A method for panoramic display of spherical enclosed cockpit based on double projection transformation as mentioned above, further, S01. In the image acquisition, in addition to the M groups of images used to project on the cylinder, it also includes a top image and a bottom image , the images are collected by the up-shooting camera and the down-shooting camera respectively.

As mentioned above, a spherical closed cockpit panorama display method based on double projection transformation, further, the M groups of images in S01 image acquisition are captured by a three-axis pan/tilt head equipped with a common lens camera and other time slots.

As mentioned above, a spherical closed cockpit panorama display method based on double projection transformation, further, the cylindrical projection transformation described in S02 panorama splicing specifically includes: projecting the M groups of images on the side of the cylinder, and then adding the image Linear fusion is performed with the edge of the image to obtain a cylindrical surround image.

As mentioned above, a spherical closed cockpit panorama display method based on double projection transformation, further, the cross-sectional radius of the cylinder is the focal length of the camera plus the distance from the lens to the center of the circle.

A kind of spherical enclosed cockpit panorama display method based on double projection transformation as mentioned above, further, S02. The spherical fitting projection transformation described in panorama stitching specifically includes: taking the spherical surface as the shadow-bearing surface, and placing the cylindrical surface and the cylindrical surface up and down The bottom surface is projected onto the spherical surface tangent to the cylindrical surface to obtain a spherical panoramic image.

As mentioned above, a spherical closed cockpit panoramic display method based on double projection transformation, further, the spherical fitting projection transformation process adopts the equiangular positive axis cylindrical back projection transformation in the low latitude area of the spherical surface, and adopts the high latitude area. Spherical projection transformation.

As described above, a spherical closed cabin panoramic display method based on double projection transformation, further, in S03 spherical display, the polygonal image of the spherical segmented part includes triangles, parallelograms, and regular pentagons.

As mentioned above, a spherical closed cockpit panorama display method based on double projection transformation, further, S03. In spherical display, the pixels are addressed in a chain sequence, and the address of each pixel is encoded from the low order to the high order.

As mentioned above, a spherical closed cockpit panorama display method based on double projection transformation, further, the chain sequence includes: the next address points to a pixel adjacent to a row; if the currently identified pixel address reaches the boundary of a row, Then the next address points to the pixel on the same side boundary of the next row.

A spherical closed cockpit panoramic display method based on double projection transformation as mentioned above, further, in S03 spherical display, the inner spherical display system includes a camera group, an image processor, an address converter, and a signal controller, A set of screen drivers and a set of polygon screens, where:

The camera group outputs multiple images to the image processor;

The image processor outputs the image information to the address converter, and outputs the pixel data signal to the screen driver;

The address converter is used to convert the image address format and output the address data signal to the screen driver at the same time;

The signal controller outputs the screen synchronization signal to the image processor, the address converter and the screen driver;

The luminance-color equalizer outputs luminance-color calibration signals to the screen driver, and receives luminance-color information as feedback from a group of polygonal screens;

a set of screen drivers for driving the polygon screen;

A set of polygonal screens that are stitched together into a spherical-like display that displays panoramic images.

As mentioned above, a spherical closed cabin panoramic display method based on double projection transformation, further, the shape of the polygonal screen includes triangle, parallelogram and regular pentagon.

Compared with the prior art, the present invention adopts the above technical scheme, and has the following technical effects:

1. The present invention adopts double projection transformation to realize spherical projection, and adopts different projection algorithms according to the latitude, which greatly reduces the degree of distortion of the image in the projection process; the present invention divides the spherical image into polygonal images for storage, and changes the sequential process into It is a parallel process, which is convenient for storage and reading, and improves the loading speed; the present invention addresses the pixels according to the chain sequence, and can adapt to any polygonal screen; Line synchronization signal, only use pixel signal and field synchronization signal to control the screen to refresh the image; on the other hand, it can adapt to arbitrary polygon screen drive, and realize arbitrary polygon splicing spherical display.

2. Compared with the traditional scheme of splicing multiple sets of screens to form a large screen, the present invention can realize the inner spherical display with large curvature and equal viewing distance; compared with the virtual reality display technology based on VR equipment, the present invention does not require users to wear equipment At the same time, it can provide a 360° wide field of view, and all polygonal screens are loaded and displayed at the same time, which solves the problem of vertigo caused by delay.

3. The present invention provides a complete set of methods for image acquisition, image transformation, and image display for a splicing-style panoramic display system. Overcome the shortcomings of the existing equipment and provide a new universal panoramic display method.

Description of drawings

Figure 1 is an overall flow chart.

FIG. 2 is a schematic diagram of the structure of a multi-channel camera group when M=3.

FIG. 3 is a schematic diagram of the structure of a three-axis pan/tilt of a rotating camera.

FIG. 4 is a top view of the mapping relationship from a plane image to a cylindrical image by a cylindrical projection transformation algorithm.

FIG. 5 is a 3D schematic diagram of a mapping relationship between a plane image and a cylindrical image by a cylindrical projection transformation algorithm.

FIG. 6 is a 3D schematic diagram of the mapping relationship between a cylindrical image and a spherical image by the spherical fitting projection transformation algorithm.

FIG. 7 is a 3D schematic diagram of the mapping relationship in the high latitude region of the spherical fit projection transformation algorithm.

FIG. 8 is a schematic diagram of a spherical pentagonal screen and a chain scanning mode of a triangular screen.

FIG. 9 is a schematic diagram of the relationship between the modules of the inner spherical display system.

Detailed ways

Below in conjunction with accompanying drawing, the technical scheme of the present invention is described in further detail:

It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with their meanings in the context of the prior art and, unless defined as herein, are not to be taken in an idealized or overly formal sense. explain.

The method provided by the present invention is applicable to a variety of hardware solutions, and two of the hardware solutions are exemplarily given here as examples, but are not limited to the scope of the hardware solutions. Figure 1 shows that the image acquisition part of the method includes two methods: multi-channel camera group acquisition and rotary camera acquisition, and any one of the image acquisition methods can be selected according to needs.

Embodiment 1. Using a multi-channel camera group for image acquisition

FIG. 2 is a schematic diagram of the structure of a multi-channel camera group when M=3. In this embodiment, M groups (M≧3) of cameras with a viewing angle α are evenly distributed on the circumference forming an included angle β, and the viewing angle α must be greater than the included angle β. If necessary, an up-shooting camera and a down-shooting camera are respectively added up and down the center of the circle. Among them, when 3≤M≤6, the cameras distributed on the circumference use wide-angle lenses; at this time, the captured images need to undergo barrel distortion correction to correct the perspective distortion of the captured images caused by the wide-angle lens; when M>6, use regular lens. (r: the distance from the lens to the center of the circle; f: the focal length of the camera; R=r+f: the radius of the above circle)

The relationship between the distance r from the lens to the center of the circle, the focal length of the camera f, the radius R of the above circle, the camera angle of view α, the included angle β and the number of camera groups M is:

其中

in

In this embodiment, M=3, that is, it includes three wide-angle cameras installed at a 120-degree angle on the circumference; this embodiment also includes two ordinary cameras mirrored above and below the center of the circle as up-cameras and down-cameras. FIG. 2 exemplarily shows a schematic structural diagram when M=3.

At this time, M images and two bottom images of A (top) and B (bottom) can be collected at one time; after multiple acquisitions, M images and two bottom images of A (top) and B (bottom) can be obtained. The M group images are first transformed by cylindrical projection and projected onto the side of a cylinder with a cross-sectional radius R (the radius R of the cylindrical cross-section is equal to the focal length f of the camera plus the distance r from the lens to the center of the circle). Then, the spherical fitting projection transformation algorithm is applied to the cylindrical image to complete the process of projecting the cylindrical image to the spherical surface.

(1) Cylindrical projection transformation algorithm

FIG. 4 and FIG. 5 are respectively a top view and a 3D schematic diagram of a mapping relationship between a plane image and a cylindrical image in the cylindrical projection transformation algorithm. First, perform cylindrical projection transformation on M plane images. In the Cartesian rectangular coordinate system, the plane image (x, y, z) is projected onto the cylindrical surface, z=-R, the cylindrical surface can be expanded into a plane along the generatrix, and the plane rectangular coordinate system is established, then the shadow-bearing surface can be Denoted as (x', y'). When M=3, the schematic plan view is shown in FIG. 4 , and the 3D schematic diagram is shown in FIG. 5 . The camera's field of view ABB'A' is the surface to be projected, its width is W, its height is H, the cylinder DEE'D' is the shadow-bearing surface, and the cylinder radius is R (R=r+f, r: the distance from the lens to the center of the circle , f: camera focal length), and its mapping relationship is expressed as follows:

The M groups of images after cylindrical projection transformation meet the principle of visual consistency. After extracting image features from adjacent images, performing image registration in the overlapping domain, and finally stitching images in a fade-in and fade-out manner, so that M groups of images After the image is linearly fused, a 360° cylindrical surround image can finally be obtained.

(2) Spherical Fitting Projection Transformation Algorithm

The image that has undergone cylindrical transformation is only a cylindrical panoramic image, and a spherical panoramic image can only be obtained after spherical transformation. The equiangular positive-axis cylindrical backprojection transformation is used to fit the cylindrical surface to the spherical surface. Deformation distortion occurs, and the degree of deformation increases with the increase of latitude. Generally, in order to prevent serious distortion at high latitudes, the latitude should be lower than 60 degrees, that is, θ>60°. Therefore, the bearing sphere is divided into three regions A, B, and C according to the latitude, as shown in Figure 6. Among them, the south latitude θ degrees to the north latitude θ degrees is the low latitude B area, and the equiangular positive axis cylindrical back projection transformation is used; the north latitude θ degrees above is the high latitude area A, and the south latitude θ degrees is the high latitude area C, both of which use spherical projection. transform.

The specific spherical fitting projection transformation algorithm is as follows:

)，其中

为球心与球面一点连线向赤道面上投影所形成的夹角，θ为球心与球面一点连线向子午面投影形成的夹角。球面半径为R，待投影面为圆柱面，圆柱面与球面切于赤道线，柱面轴心与球面地轴重合，如图6。柱面可沿任意一条母线展开成一二维平面。如图5所示，以赤道线为x轴，母线对应经线为y轴建立平面直角坐标系，即柱面用(x,y)表示。则等角正轴圆柱反投影变换公式如下：First, the equiangular positive-axis cylindrical backprojection transformation is adopted in the low latitude area: in the Cartesian Cartesian coordinate system, the shadow-bearing surface is a spherical surface, and the spherical surface is represented by latitude and longitude lines as (

),in

is the angle formed by the projection of the line connecting the center of the sphere and a point on the sphere to the equatorial plane, and θ is the angle formed by the projection of the line connecting the center of the sphere and a point on the sphere to the meridian plane. The radius of the spherical surface is R, the projected surface is a cylindrical surface, the cylindrical surface and the spherical surface are tangent to the equator, and the axis of the cylindrical surface coincides with the ground axis of the spherical surface, as shown in Figure 6. The cylinder can be expanded into a two-dimensional plane along any generatrix. As shown in Figure 5, a plane rectangular coordinate system is established with the equator as the x-axis and the corresponding meridian of the generatrix as the y-axis, that is, the cylinder is represented by (x, y). Then the formula for the backprojection transformation of the equiangular positive axis cylinder is as follows:

)，球面半径为R；待投影面为平面，待投影面与切于球面极点的平面A平行，与球面球心距离为h。在待投影面上建立正交平面直角坐标系(X-O-Y)，即承影面可用(x,y)表示，如图7所示。则球面投影变换公式如下：Then, the spherical projection transformation is used in the high latitude area: in the Cartesian Cartesian coordinate system, the shadow-bearing surface is a spherical surface, and the spherical surface is represented by latitude and longitude lines as (

), the radius of the spherical surface is R; the projected surface is a plane, the projected surface is parallel to the plane A tangent to the spherical pole, and the distance from the spherical center of the spherical surface is h. An orthogonal plane rectangular coordinate system (XOY) is established on the surface to be projected, that is, the shadow-bearing surface can be represented by (x, y), as shown in Figure 7. Then the spherical projection transformation formula is as follows:

Bilinear interpolation is performed between the projected image and the to-be-projected image, so that the pixels of the to-be-projected image can be evenly distributed on the image-bearing surface. Finally, the three projection domains are spliced into a complete spherical panorama, and the three must also meet the principle of visual consistency, that is, the viewing angle α of the upper camera and the lower camera must meet:

Among them, H=2h

After the above steps, we get a spherical panorama image.

The spherical panorama image needs to be segmented according to the splicing method of the spherical display, re-addressed, and stored in slices. The spherical image segmentation method includes polygonal segmentation methods such as triangle segmentation, parallelogram segmentation, and regular pentagon segmentation, and the spherical image is divided into N polygonal images.

In order to facilitate the display of special-shaped screens, the pixel data addresses need to be reordered in a chained order before the divided images are stored. The chain sequence means that the next address points to a pixel adjacent to a row; if the currently identified pixel address reaches the boundary of a row, the next address points to a pixel on the same side boundary of the next row. Within the address of a pixel, the conversion sequence is from low to high. Finally, the two-dimensional matrix encoding is converted into a chained array encoding, as shown in Figure 8.

The corresponding screen splicing methods can be divided into triangle splicing, parallelogram splicing, regular pentagon splicing and other polygon splicing methods to form a spherical display. When the screen is refreshed and displayed, the corresponding image blocks are read from the buffer area, and the pixel units are sequentially lit up according to the chain sequential scanning method.

The composition of the inner spherical display system is shown in Figure 9, including an image processor for spherical fitting, spherical segmentation, image storage and other processing for the input panorama; a brightness and color equalizer to correct the color between each screen and brightness unevenness, so that the overall brightness and color of the splicing screen are consistent; a signal control device controls the screen refresh display and synchronizes the timing of multiple screens; an address converter converts the two-dimensional matrix address code into a chain address coding; a set of screen drivers, which are used to drive special scanning methods for special-shaped polygon screens; a set of polygon screens, which are used for splicing spherical displays to display panoramic images.

Embodiment 2. Using a rotating camera to capture images

As shown in Figure 3, the structure of the rotating camera is a three-axis brushless PTZ equipped with a single camera, and the camera lens is an ordinary lens. Among them, the Roll axis and Pitch axis of the gimbal can stabilize the camera to prevent jelly from appearing in the shooting picture; the Yaw axis can be rotated 360 degrees. Each time the Yaw axis rotates through the β angle, the camera captures an image, rotates one circle to capture M images, and finally generates a panoramic image through image stitching. After rotating for many times, a total of M groups of images are left on the M camera positions to generate multiple panoramic images. The specific steps of subsequent image stitching and spherical display are the same as those described in the first embodiment.

The above are only some embodiments of the present invention. It should be pointed out that for those skilled in the art, without departing from the principles of the present invention, several improvements and modifications can be made. It should be regarded as the protection scope of the present invention.

Claims (8)

1. a spherical closed cockpit panorama display method based on double projection transformation, is characterized in that, step comprises: S01, image acquisition: acquire at least M groups of images; S02, panoramic stitching: the M groups of images are subjected to projection transformation to obtain a complete spherical panoramic image; the projection transformation includes cylindrical projection transformation and spherical fitting projection transformation; S03, spherical display, specifically including: Spherical segmentation: segment the spherical panoramic image to obtain N equal-sized polygon images; Block storage: store the polygon image in the buffer in blocks; Chain addressing: address the pixels in a chain sequence within each polygon image and re-encode; Chain Scan: Scan and drive the physical screen in a chain sequence; Wherein, the chain sequence includes: the next address points to a pixel adjacent to the same row; if the currently identified pixel address reaches the boundary of a row, the next address points to the pixel on the same side boundary of the next row; Block display: read the storage content of the polygon image in the buffer, and display it on the physical screen corresponding to this area; In the above spherical display, the pixels are addressed in a chained sequence, and the address of each pixel is coded from low-order to high-order. 2 . The method for displaying panorama of spherical enclosed cockpit based on double projection transformation according to claim 1 , wherein, in the image acquisition described in step S01 , in addition to the M groups of images used for projecting on the cylinder, 2 . It includes a top surface image and a bottom surface image, and the images are respectively captured by the up-camera and the down-camera. 3. a kind of spherical closed cockpit panorama display method based on double projection transformation according to claim 1, is characterized in that, the cylindrical projection transformation of the panorama stitching described in step S02 specifically comprises: projecting M groups of images to the cylindrical side surface , and then linearly fuse the image with the edge of the image to obtain a cylindrical surround image. 4 . The method for displaying panoramic view of a spherical closed cockpit based on dual projection transformation according to claim 3 , wherein the cross-sectional radius of the cylinder is the focal length of the camera plus the distance from the lens to the center of the circle. 5 . 5. a kind of spherical enclosed cockpit panorama display method based on double projection transformation according to claim 1, is characterized in that, S02. The spherical fitting projection transformation described in panorama splicing specifically comprises: taking the spherical surface as the shadow-bearing surface, The cylindrical surface and the upper and lower bottom surfaces of the cylindrical surface are projected onto a spherical surface tangent to the cylindrical surface to obtain a spherical panoramic image. 6. A spherical closed cockpit panorama display method based on double projection transformation according to claim 5, characterized in that, the spherical fitting projection transformation process adopts isometric positive axis cylindrical back projection transformation in the low latitude region of the spherical surface , and spherical projection transformation is used in high latitude regions. 7 . The method for displaying panoramic view of spherical closed cockpit based on double projection transformation according to claim 6 , wherein the low latitude area is an area with a latitude range of 0° to 60° on the spherical surface, and a high latitude area. 8 . A region is an area on the sphere whose latitude ranges from 60° to 90°. 8 . The method for displaying a panorama of a spherical closed cockpit based on dual projection transformation according to claim 1 , wherein, in the spherical display described in step S03 , the inner spherical display system comprises a camera group, an image processor, an address converter , a signal controller, a set of screen drivers and a set of polygon screens, where: The camera group outputs multiple images to the image processor; The image processor outputs the image information to the address converter, and outputs the pixel data signal to the screen driver; The address converter is used to convert the image address format and output the address data signal to the screen driver at the same time; The signal controller outputs the screen synchronization signal to the image processor, the address converter and the screen driver; The luminance-color equalizer outputs luminance-color calibration signals to the screen driver, and receives luminance-color information as feedback from a group of polygonal screens; a set of screen drivers for driving the polygon screen; A set of polygonal screens that are stitched together into a spherical-like display that displays panoramic images.

Images