CN117793317A - A multi-sensor Kalman fusion dynamic projection method and device for special-shaped surfaces - Google Patents

Abstract

The invention discloses a multi-sensor Kalman fusion type special-shaped surface dynamic projection method and a device, comprising the following steps: acquiring real-time pose information of a visual special-shaped surface and real-time pose information of an inertial positioning special-shaped surface; fusing the real-time pose information of the visual special-shaped surface and the real-time pose information of the inertial positioning special-shaped surface to obtain fused pose information; calculating to obtain a projector image according to the fusion pose information; the projector projects the projector image, and the dynamic projection of the special-shaped surface is realized. By adopting the technical scheme of the invention, the special-shaped surface projection is easy to realize; meanwhile, when the illumination condition changes severely, the real-time pose of the special-shaped surface is easy to obtain.

Description

A multi-sensor Kalman fusion dynamic projection method and device for special-shaped surfaces

Technical field

The invention belongs to the field of dynamic projection technology, and in particular relates to a multi-sensor Kalman fusion dynamic projection method and device for special-shaped surfaces.

Background technique

Dynamic projection has broad application prospects in cultural tourism performances and exhibition displays. At present, the common projection display method is that the projector is stationary and the projection surface is a regular screen such as a flat screen or a curved screen. Dynamic projection means that the position of the projector is fixed and the projection surface moves. Dynamic projection makes the creation of cultural and tourism performances, exhibitions and other programs more flexible, giving the audience a more shocking viewing experience.

During dynamic projection, the position of the special-shaped projection surface changes, so it is necessary to obtain the pose information of the projection surface in real time. In addition, the surface of the special-shaped surface is uneven, and the homography transformation used in regular screen projection cannot be used to achieve projection geometric correction. Compared with the projection of regular screens (flat screens, curved screens, etc.), special-shaped surface projection is significantly more difficult. At the same time, when the lighting conditions change drastically, the real-time pose of the special-shaped surface may fail to be obtained.

Contents of the invention

The technical problem to be solved by the present invention is to provide a method and device for dynamic projection of irregular surfaces by multi-sensor Kalman fusion.

In order to achieve the above objects, the present invention adopts the following technical solutions:

A multi-sensor Kalman fusion dynamic projection method of special-shaped surfaces, including:

Obtain real-time position and posture information of visual profiled surfaces and real-time position and posture information of inertial positioning profiled surfaces;

Fusion of the real-time pose information of the visual irregular surface and the real-time pose information of the inertial positioning irregular surface to obtain the fused pose information;

Based on the fused pose information, the projector image is calculated;

The projector projects the projector image to realize dynamic projection of special-shaped surfaces.

As a preferred option, the error state Kalman filter is used to fuse the real-time pose information of the visual irregular surface and the real-time pose information of the inertial positioning irregular surface.

As a preferred method, the projector image is calculated based on the dense point cloud on the surface of the special-shaped surface, the internal and external parameters of the projector, and the fused pose information.

As an option, the camera takes real-time images of the special-shaped surface to obtain real-time pose information of the visual special-shaped surface; rigidly fix the inertial positioning system and the special-shaped surface, and calibrate the inertial positioning system to obtain real-time pose information of the inertial positioning special-shaped surface.

The invention also provides a multi-sensor Kalman fusion special-shaped surface dynamic projection device, including:

The acquisition module is used to obtain the real-time pose information of the visual irregular surface and the real-time pose information of the inertial positioning irregular surface;

The fusion module is used to fuse the real-time pose information of the visual irregular surface and the real-time pose information of the inertial positioning irregular surface to obtain the fused pose information;

The calculation module is used to calculate the projector image based on the fused pose information;

The projection module is used for projecting a projector image to realize dynamic projection of special-shaped surfaces.

Preferably, the fusion module utilizes error state Kalman filtering to fuse the real-time pose information of the visual profiled surface and the real-time pose information of the inertial positioning profiled surface.

As an option, the calculation module calculates the projector image based on the dense point cloud on the surface of the special-shaped surface, the internal and external parameters of the projector, and the fused pose information.

As an option, the acquisition module takes real-time images of the special-shaped surface through a camera to obtain real-time pose information of the visual special-shaped surface; the acquisition module rigidly fixes the inertial positioning system and the special-shaped surface, and calibrates the inertial positioning system to obtain the real-time position of the inertial positioning special-shaped surface. posture information.

This invention obtains the real-time pose information of the visual irregular surface and the real-time pose information of the inertial positioning irregular surface; fuses the real-time pose information of the visual irregular surface and the real-time pose information of the inertial positioning irregular surface to obtain the fused pose information; according to the fused pose information, The projector image is calculated; the projector projects the projector image to realize dynamic projection of special-shaped surfaces. Using the technical solution of the present invention, it is easy to realize the projection of the special-shaped surface; at the same time, when the lighting conditions change drastically, it is easy to obtain the real-time pose of the special-shaped surface.

Description of drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are only These are embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on the provided drawings without exerting creative efforts.

FIG1 is a flow chart of a method for dynamic projection of a profiled surface using multi-sensor Kalman fusion according to an embodiment of the present invention;

Figure 2 is a flow chart of another multi-sensor Kalman fusion dynamic projection method of irregular surfaces according to an embodiment of the present invention;

Figure 3 is a schematic structural diagram of a special-shaped surface dynamic projection system diagram of multi-sensor Kalman fusion according to an embodiment of the present invention;

Figure 4 is a projector-camera system calibration hardware configuration diagram;

FIG5 is a schematic diagram of the principle of three-dimensional reconstruction;

Figure 6 is a schematic diagram of the inertial positioning system.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

In order to make the above objects, features and advantages of the present invention more obvious and understandable, the present invention will be described in further detail below with reference to the accompanying drawings and specific embodiments.

Example 1:

As shown in FIG1 , an embodiment of the present invention provides a multi-sensor Kalman fusion method for dynamic projection of a profiled surface, comprising:

Obtain real-time position and posture information of visual profiled surfaces and real-time position and posture information of inertial positioning profiled surfaces;

Fusion of the real-time pose information of the visual irregular surface and the real-time pose information of the inertial positioning irregular surface to obtain the fused pose information;

Based on the fused pose information, the projector image is calculated;

The projector projects the projector image to realize dynamic projection of special-shaped surfaces.

As an implementation method of the embodiment of the present invention, the error state Kalman filter is used to fuse the real-time pose information of the visual irregular surface and the real-time pose information of the inertial positioning irregular surface.

As an implementation method of the embodiment of the present invention, the projector image is calculated based on the dense point cloud on the surface of the special-shaped surface, the internal and external parameters of the projector, and the fused pose information.

As an implementation method of the embodiment of the present invention, a camera is used to capture the image of the special-shaped surface in real time to obtain the real-time pose information of the visual special-shaped surface; the inertial positioning system and the special-shaped surface are rigidly fixed, and the inertial positioning system is calibrated to obtain the inertial positioning Real-time pose information of special-shaped surfaces.

Example 2:

As shown in FIGS. 2 and 3 , an embodiment of the present invention provides a multi-sensor Kalman fusion method for dynamic projection of a profiled surface, comprising:

Step S1: Rigidly fix the projector and camera to obtain the internal and external parameters of the camera and the internal and external parameters of the projector;

Step S2: The projector projects a set of structured light images, and the camera captures the structured light image modulated by the special-shaped surface;

Step S3: Obtain the dense point cloud on the surface of the special-shaped surface;

Step S4: The camera captures images of the special-shaped surface in real time to obtain real-time pose information of the visual special-shaped surface;

Step S5: Rigidly fix the inertial positioning system and the special-shaped surface, and calibrate the inertial positioning system to obtain real-time pose information of the inertial positioning special-shaped surface;

Step S6: Use the error state Kalman filter to fuse the real-time pose information of the visual irregular surface and the real-time pose information of the inertial positioning irregular surface to obtain the fused pose information;

Step S7: Calculate the projector image based on the dense point cloud on the surface of the special-shaped surface, the internal and external parameters of the projector, and the fused pose information;

Step S8: The projector projects the projector image to realize dynamic projection of the special-shaped surface.

As an implementation method of the embodiment of the present invention, in step S1, as shown in Figure 4, the calibration of the projector-camera system includes the calibration of the internal parameters of the camera and the projector and the posture of the projector relative to the projection screen (or called projection instrument external parameters) calibration. Calibration of internal and external parameters of the projector is the basis of projection geometric correction, and calibration of internal parameters is the basis of calibration of external parameters. The projector does not have the ability to actively acquire images, so a camera is needed to achieve internal and external parameter calibration.

As shown in Fig. 3, the projector and the camera are rigidly fixed to form a projector-camera system. The projector-camera system is oriented toward the spatial reference plane with the checkerboard image attached, and the angle of the projector-camera system is adjusted so that the projection area does not overlap with the checkerboard image and the camera field of view can completely cover the projection area and the checkerboard image. When calibrating the camera, the projector is turned off, and the checkerboard image is photographed and calibrated by the camera to obtain the camera intrinsic parameter matrix _Kc and the rotation matrix _Rc and translation matrix _Tc of the camera extrinsic parameters.

The process of obtaining the rotation matrix R _c of the camera's external parameters is as follows: the rotation matrix is a sequence composite of three basic rotations. The rotations about the x, y, and z axes of the right-handed Cartesian coordinate system are called R _x , R _y , and R _z respectively. R _x can be defined as θ is the angle of rotation around the x-axis. In the same way, R _y and R _z can be obtained, and the rotation matrix R _c is obtained by multiplying the three basic rotations.

The process of obtaining the translation matrix T _c of the camera's external parameters is as follows: the translation matrix T _c is the translation distance along the x, y, and z axes of the right-hand Cartesian coordinate system, and t _x is the translation distance along the positive direction of the x axis, so The translation matrix T _c can be expressed as

Under the condition that the corresponding relationship between the three-dimensional space point X _w and the two-dimensional image point x _w is known, the values of the internal and external parameters K _c , R _c and T _c of the camera are obtained. The calculation formula is:

x _w =PX _w =K _c [R _c |T _c ]X _w

The camera is calibrated using a customized checkerboard calibration plate. The size of the checkerboard on the calibration plate can be obtained by measurement, and its corresponding two-dimensional image coordinates can be obtained by extracting image corner points.

After obtaining the internal parameters of the camera, based on the corresponding relationship between the known three-dimensional space point X _w and the two-dimensional image point x _w , a linear or nonlinear algorithm is used to obtain the camera external parameters.

The projector can be regarded as a dual system of the camera and is calibrated using the same imaging model as the camera. However, since the projector does not have the ability to actively acquire images, it is necessary to calibrate the projector with the help of a calibrated camera. When calibrating the camera, the correspondence between the three-dimensional space points in the world coordinate system and the two-dimensional coordinate points in the image coordinate system is achieved through manual measurement and feature point identification. In the projector system, the two-dimensional coordinate points in the image coordinate system are extracted by identifying feature points in the same way as the camera method, but the position of the three-dimensional points projected into space is difficult to measure. Therefore, the projector is associated with the camera, and the spatial measurement capability of the camera is used to indirectly calibrate the projector.

After the camera is calibrated, remove the checkerboard image pasted on the spatial reference plane, then turn on the projector to project the checkerboard image. Assume that the two-dimensional coordinates of the corner point of the checkerboard image in the imaging plane of the projector are x _w , its corresponding three-dimensional space point on the spatial reference plane is X _w , and the coordinates of the two-dimensional image captured by the camera are x _c . x _w and x _c can be obtained through the image corner point extraction method, and X _w is calculated and obtained by relying on the calibrated camera.

Since the corresponding relationship between the two-dimensional coordinate point _xw of the projector image and its corresponding three-dimensional space point _Xw is established, the camera calibration method can be used to calibrate the projector to obtain the intrinsic and extrinsic parameters _Kp , _Rp and _Tp of the projector. After the projector is calibrated, the mapping relationship between the projector image coordinates and the camera image coordinates is further calculated. Assuming that the three-dimensional coordinates of the same space point _Xw in the camera and projector coordinate systems are _Xc and _Xp respectively, the coordinate transformation relationship between them can be expressed as follows:

Eliminating X _w in the above system of equations, we can get:

X _C ＝R _CP X _P +T _CP

Among them: R _CP =R _C R _P ^-1 ; T _CP = _TC -R _C R _P ^-1 T _P .

As an implementation of an embodiment of the present invention, in step S2, an active vision method based on structured light is adopted, and a projector projects a set of Gray code encoded structured light images onto the profiled surface; and a camera captures the structured light image modulated by the profiled surface.

As an implementation method of the embodiment of the present invention, in step S3, the three-dimensional surface of the special-shaped surface is calculated based on the structured light image projected by the projector, the modulated structured light image captured by the camera, and the internal and external parameter matrix of the projector-camera system. The coordinate P of the point is used to obtain a dense point cloud on the surface of the special-shaped surface, as shown in Figure 5.

The structured light image modulated by the profiled surface taken by the camera is decoded by Gray code to obtain the correspondence between the structured light image and the structured light image modulated by the profiled surface, as shown in Figure 4. _PL is the pixel of the structured light image projected by the projector, _PR is the pixel corresponding to the modulated structured light image taken by the camera, and P is the point in the world coordinate system where the structured light image projected by the projector is projected on the profiled surface. Given the internal and external parameters of the projector _PL and _PR and the internal and external parameters of the camera, calculate P:

Among them (X, Y, Z) are the three-dimensional coordinates of point P, that is, the coordinates of the special-shaped surface point cloud to be solved; K _c is the internal parameter matrix of the camera; R _c and T _c are the external parameter matrices of the camera; K _p is the internal parameter of the projector matrix; R _p and T _p are the external parameter matrices of the projector; (up _, v _p ) are the coordinates of the projector pixel P _L , and ( _uc , v _c ) are the coordinates of the camera pixel P _R.

As an implementation mode of the embodiment of the present invention, in step S4, the embodiment of the present invention uses the SURF algorithm to extract feature points. The SURF algorithm can more accurately extract feature points on edges and weak textures. The SURF algorithm extracts The feature points are more dispersed, which is beneficial to subsequent calculation of special-shaped surface pose information.

After the feature point extraction is completed, the FLANN feature point matching method is then used to match the feature points to obtain matching point pairs. The FLANN matching algorithm is fast in operation.

Assume that the obtained matching point pair is p1 and p2, and the camera internal parameter matrix is K _c . Using epipolar constraints, find the rotation matrix R _x and translation matrix t _x of the special-shaped surface relative to the previous moment, and use matrix decomposition:

Find the fundamental matrix (Fundamental Matrix) F:

Given the camera internal parameter matrix K _c , further find the essential matrix (Essential Matrix) E:

The eight-point method is used to calculate the rotation matrix R _x and translation matrix t _x of the irregular surface at the current moment, and obtain the real-time pose information of the visual irregular surface.

Due to the influence of lighting and other factors, the images captured by the camera contain noise. Preprocessing the images collected by the camera can improve the accuracy of image feature point matching. This embodiment uses Gaussian bilateral filtering to denoise:

in:

Among them, x is the current point position; y is the point in the s*s area; I _x and I _y are the pixel values of the current point; G _σd is the spatial neighborhood relationship function; ‖x-y‖ is the spatial distance; G _σr is Gray value similarity relationship function; σ _d and σ _r are Gaussian standard deviations.

The feature points of images after Gaussian bilateral filtering are robust, which is conducive to feature point extraction and matching of camera images at adjacent moments.

As an implementation method of an embodiment of the present invention, in step S5, the inertial positioning system integrates multiple sensors such as gyroscopes, accelerometers, encoders, etc. After the inertial positioning system is powered on, it is automatically initialized and uses its own center as the coordinate origin to obtain coordinates in the x-direction and y-direction as shown in Figure 6. The coordinate system of the inertial positioning system is defined as an absolute coordinate system. Once the inertial positioning system is installed on the special-shaped surface, this coordinate system is determined after the inertial positioning system is powered on. Since the inertial positioning system supports real-time updating of angles and coordinates, the coordinate system of the inertial positioning system will change each time the data is updated.

Since the coordinate system of the inertial positioning system and the coordinate system of the camera do not coincide, the inertial positioning system and the profiled surface need to be calibrated before use. The calibration calculation expression is as follows:

Among them, x, y are the coordinates read by the inertial positioning system, t = [t ₁ , t ₂ , t ₃ ] ^T is the translation matrix of the special-shaped surface, and the translation matrix of the special-shaped surface uses feature point matching through camera feedback. And the epipolar constraint algorithm is obtained, and the calibration parameter A is obtained through the above formula:

At subsequent intervals, the inertial positioning system will obtain the coordinates x and y of the special-shaped surface in real time. It is known that the inertial positioning system transmits the coordinates x and y of the special-shaped surface in real time. According to the calibration parameter A, the special-shaped surface relative to the camera at this time is calculated. Surface rotation matrix R' and translation vector t'.

As an implementation method of the embodiment of the present invention, in step S6, the method for estimating the pose of the special-shaped projection surface based on camera feedback has no cumulative error during use, and can obtain the pose of the special-shaped projection surface relatively accurately. However, due to complex calculations, the time delay of the visual pose estimation method based on camera feedback cannot meet the needs of dynamic projection alone. The special-shaped projection surface pose estimation method based on inertial sensors has good real-time performance and short calculation time. The real-time performance meets the needs of dynamic projection. However, inertial sensors will produce cumulative errors over time, so it needs to be combined with pose estimation based on camera feedback. method to correct. The present invention uses an error state Kalman filter to perform data fusion on visual pose and inertial sensor pose.

(1) Construct a filter system

Define the state vector δX = [δ _pi δv δθ _i δb _a δb _ω ] ^T .

Among them, the state vector δX is the error vector solved by the inertial positioning system. Because the inertial positioning system and the profiled surface are rigidly fixed, the error solved by the inertial positioning system can be used to represent the errors of various parameters of the profiled surface: _δpi is the solved position error of the profiled surface, _δθi is the solved attitude error of the profiled surface, _δv is the velocity error of the profiled surface in the current state, δba is the zero bias error of the accelerometer of the inertial positioning system, and _δbω is the zero bias error of the gyroscope of the inertial positioning system.

Then the state equation of the system is:

δX _k ＝F _k-1 δX _k-1 +B _k-1 w _k

in, is the state transition matrix;/> a and ω are the outputs of the accelerometer and gyroscope respectively; T is the Kalman filter period; R is the rotation matrix; i is the unit matrix. Define system noise/> w _k ~N(0,Q _k ), Q is the system noise covariance matrix. n _a and n _ω are the white noise of the accelerometer and the white noise of the gyroscope respectively, They are the Gaussian random walk noise of the accelerometer and the Gaussian random walk noise of the gyroscope. The operator [·] _× indicates that each element in the vector forms an antisymmetric matrix.

Define the observation vector Z = [δp _v δθ _v ] ^T .

Among them, δp _v is the observation position error, and δθ _v is the observation attitude error. δp _v and δθ _v are calculated from the following formula:

Among them, p _i and R _i are the position and rotation matrices solved by the inertial positioning system, and p _v and R _v are the position and rotation matrices estimated by the camera.

The observation equation of the system is:

Z _k =H _k δX _k +C _k n _k

Among them, the observation matrix Observation noise/> The observation noise satisfies E(n _k )=0,n _k ~N(0,N _k ), and N represents the observation noise covariance matrix.

(2) Initialization

The state quantity δX ₀ is initially 0, and the initial value of each noise power is set. The process noise and observation noise generally remain unchanged during the iteration.

(3) Forecast update

Kalman filter prediction process:

Kalman filter update process:

Where P _k is the covariance matrix of the system. _{K k} is the Kalman gain.

(4) Calculate the posterior pose

According to the posterior state quantity, update the posterior pose and output the position after Kalman filtering and rotation matrix/>

The posterior pose value obtained through Kalman filtering is integrated with the data obtained from the inertial positioning system and camera feedback. The pose is more accurate through multi-sensor data fusion, which can improve the accuracy of subsequent dynamic projection of special-shaped surfaces.

As an implementation method of the embodiment of the present invention, in step S7, the dense point cloud information on the special-shaped surface obtained by three-dimensional reconstruction, the internal and external parameters of the projector calibrated by the projector-camera system, and the fused pose information are used. Realize pre-distortion of the image to be projected, input the pre-distorted image into the projector, and calculate the projector image.

Calculate the rotation matrix R _x of the obtained special-shaped surface relative to the previous moment, the translation matrix t _x and the external parameters of the camera to obtain the motion parameters R and T of the projection screen relative to the projector:

R＝R _c R _x , T＝T _c +t _x

The conversion formula from the 3D point cloud of the irregular surface to the 2D coordinates of the projector image is as follows:

_xw ＝ _PXw ＝ _Kp [R|T] _Xw

Among them, _Xw is the three-dimensional point cloud coordinate; R, T are the rotation matrix and translation matrix of the projector relative to the projection surface at the current moment, _Kp is the internal parameter matrix; _xw is the two-dimensional coordinate of the projector image.

According to the above formula, after calculating the two-dimensional coordinates x _w of the projector image corresponding to the three-dimensional point cloud coordinate X _w , the color information of the three-dimensional point cloud at the coordinate X _w is assigned to the projector image pixel at x _w . If this operation is performed on all point clouds on the special-shaped surface, the projector image will be obtained to realize dynamic projection of the special-shaped surface.

Example 3:

An implementation example of the present invention provides a multi-sensor Kalman fusion special-shaped surface dynamic projection device, including:

The acquisition module is used to obtain the real-time pose information of the visual irregular surface and the real-time pose information of the inertial positioning irregular surface;

The fusion module is used to fuse the real-time pose information of the visual irregular surface and the real-time pose information of the inertial positioning irregular surface to obtain the fused pose information;

A calculation module, used for calculating and obtaining a projector image according to the fused posture information;

The projection module is used by the projector to project the projector image to achieve dynamic projection of special-shaped surfaces.

As an implementation method of the embodiment of the present invention, the fusion module uses error state Kalman filter to fuse the real-time pose information of the visual irregular surface and the real-time pose information of the inertial positioning irregular surface.

As an implementation method of the embodiment of the present invention, the calculation module calculates the projector image based on the dense point cloud on the surface of the special-shaped surface, the internal and external parameters of the projector, and the fused pose information.

As an implementation method of the embodiment of the present invention, the acquisition module takes real-time images of the special-shaped surface through a camera to obtain real-time pose information of the visual special-shaped surface; the acquisition module rigidly fixes the inertial positioning system and the special-shaped surface, and calibrates the inertial positioning system , obtain the real-time pose information of the inertial positioning special-shaped surface.

The above-described embodiments are only descriptions of preferred modes of the present invention and do not limit the scope of the present invention. Without departing from the design spirit of the present invention, those of ordinary skill in the art can make various modifications to the technical solutions of the present invention. All deformations and improvements shall fall within the protection scope determined by the claims of the present invention.

Claims (8)

1. A multi-sensor Kalman fusion dynamic projection method of special-shaped surfaces, which is characterized by: Obtain the real-time pose information of the visual irregular surface and the real-time pose information of the inertial positioning irregular surface; Fusion of the real-time pose information of the visual irregular surface and the real-time pose information of the inertial positioning irregular surface to obtain the fused pose information; Based on the fused pose information, the projector image is calculated; The projector projects the projector image to realize dynamic projection of special-shaped surfaces. 2. The dynamic projection method of multi-sensor Kalman fusion of irregular surfaces according to claim 1, characterized in that error state Kalman filtering is used to fuse the real-time pose information of visual irregular surfaces and the real-time posture information of inertial positioning irregular surfaces. 3. The dynamic projection method of multi-sensor Kalman fusion of special-shaped surfaces according to claim 2, characterized in that the projector image is calculated based on the dense point cloud on the surface of the special-shaped surface, the internal and external parameters of the projector and the fusion pose information. 4. The dynamic projection method of multi-sensor Kalman fusion of special-shaped surfaces as claimed in claim 3, characterized in that the real-time posture information of the visual special-shaped surface is obtained by taking real-time images of the special-shaped surface through a camera; and the inertial positioning system and the special-shaped surface are processed Rigidly fix and calibrate the inertial positioning system to obtain real-time pose information of the inertial positioning special-shaped surface. 5. A multi-sensor Kalman fusion special-shaped surface dynamic projection device, which is characterized by including: The acquisition module is used to obtain the real-time pose information of the visual irregular surface and the real-time pose information of the inertial positioning irregular surface; The fusion module is used to fuse the real-time pose information of the visual irregular surface and the real-time pose information of the inertial positioning irregular surface to obtain the fused pose information; The calculation module is used to calculate the projector image based on the fused pose information; The projection module is used by the projector to project the projector image to achieve dynamic projection of special-shaped surfaces. 6. The multi-sensor Kalman fusion special-shaped surface dynamic projection device according to claim 5, characterized in that the fusion module uses error state Kalman filtering to fuse the visual special-shaped surface real-time pose information and the inertial positioning special-shaped surface real-time pose information . 7. The multi-sensor Kalman fusion special-shaped surface dynamic projection device as described in claim 6 is characterized in that the calculation module calculates the projector image based on the dense point cloud of the special-shaped surface, the internal and external parameters of the projector and the fused posture information. 8. The multi-sensor Kalman fusion special-shaped surface dynamic projection device as claimed in claim 7, characterized in that the acquisition module captures the special-shaped surface image in real time through a camera to obtain the real-time pose information of the visual special-shaped surface; the acquisition module controls the inertial positioning system. Rigidly fix the special-shaped surface and calibrate the inertial positioning system to obtain real-time pose information of the inertial positioning special-shaped surface.