CN115866416A - Multi-channel video fusion display system of shipborne video monitoring system - Google Patents

Abstract

The invention relates to the technical field of video fusion, in particular to a multi-channel video fusion display system of a ship-borne video monitoring system. It includes: ship-borne video monitoring system, which collects video data from multiple locations on the ship through ship-borne monitoring equipment, and transmits it to the background data management platform through communication lines; the data management platform includes: video fusion system, through multiple video data Analyze the feature points to obtain the corresponding spatial position and time axis of each video data, extract multiple video images of the same time axis, and sequentially fuse them into a panoramic fusion image of multiple perspectives according to the order of adjacent spatial positions; display interactive system, Display the panoramic fusion screen of multiple perspectives, and the user can retrieve and view video data and perform multi-terminal collaborative interaction through management authority. The invention fuses multi-channel videos collected by shipboard monitoring, organically and seamlessly fuses isolated multiple cameras into panoramic images, and realizes panoramic real-time monitoring of the entire ship.

Description

A multi-channel video fusion display system for a ship-borne video surveillance system

technical field

The invention relates to the technical field of video fusion, in particular to a multi-channel video fusion display system of a ship-borne video monitoring system.

Background technique

In recent years, with the rapid development of my country's shipping industry, various ship video surveillance systems have emerged and are gradually used in water transportation management. Many shipping companies have installed video surveillance systems on board, which are mainly used for real-time monitoring of ship anti-collision, cabin cargo monitoring, and ship management.

The traditional ship monitoring solution uses several cameras to shoot scenes from different perspectives, and returns the scene information to multiple independent windows of the client to realize monitoring. Due to the complexity of the space of the ship, especially the partial overlapping of the monitoring area, the existing monitoring technology requires the monitoring personnel to monitor multiple windows, the video monitoring screen is scattered, the system is numerous and discontinuous, the overall lack of sense of space, and the difficulty of event tracing and other problems , has been unable to meet the needs of modern shipping.

Contents of the invention

The invention provides a multi-channel video fusion display system of a ship-borne video monitoring system, which aims to solve the problems of scattered pictures, weak sense of space, and difficult traceability in the existing ship monitoring.

The present invention provides a multi-channel video fusion display system of a shipborne video surveillance system, comprising:

The ship-borne video surveillance system collects video data from multiple locations on the ship through ship-mounted monitoring equipment and encodes them to form video streams, which are transmitted to the background data management platform through communication lines for data storage, video stream management, video data display and interaction;

The data management platform includes video fusion system and display interaction system;

The video fusion system, by analyzing the feature points of multiple video data, obtains the corresponding spatial position and time axis of each video data, extracts multiple video images of the same time axis, and sequentially fuses them into multiple images in the order of adjacent spatial positions. The panoramic fusion screen from the perspective of the road, the video stream transmits the video frames sequentially according to the sequence of the time axis;

The display interaction system displays the panoramic fusion screen of multiple perspectives, and the user can retrieve and view video data and perform multi-terminal collaborative interaction through management authority.

As a further improvement of the present invention, the monitoring equipment of the ship-borne video monitoring system includes a ship-specific network video recorder, a monitoring camera, and a monitoring probe.

As a further improvement of the present invention, the communication line of the ship-borne video monitoring system includes a mobile transmission network of the front-end monitoring equipment and an Ethernet or LAN network of the background data management platform.

As a further improvement of the present invention, the video fusion system includes:

The feature acquisition module is used to obtain the internal and external parameters of multiple monitoring devices, extract the corresponding spatial position marks of each monitoring device on the ship in the internal and external parameters, and arrange the video images formed by multiple monitoring devices in the order of the spatial position marks, To be spliced and fused; obtain video data of multiple monitoring devices, extract the time axis of each video data, and use the frame pictures within the same time axis as the same group of video pictures to be spliced and fused.

As a further improvement of the present invention, the video fusion system includes:

The analysis and judgment module is used to assign a value to the time axis of each video data by setting a time axis deviation threshold, if the deviation of the time axis assignment of each video data is less than or equal to the time axis deviation threshold, then the time axis of each video data is located at Within the range of the same time axis; if the difference between the time axis assignments of the two video data is greater than the time axis deviation threshold, the video data corresponding to the time axis is discarded.

As a further improvement of the present invention, the video fusion system includes:

The fusion module is used to map the video images arranged in the order of adjacent spatial positions to the panoramic pixel coordinate system for splicing and fusion through the pre-calibrated panoramic pixel coordinate system, and perform filtering processing on the panoramic fusion picture to smooth the fusion of two adjacent frames Brightness between frames.

As a further improvement of the present invention, the data management platform also includes:

The front-end data cache unit is used to set multiple cache partitions, correspondingly store the video streams collected by each monitoring device, and after summarizing multiple video streams, output them to the video fusion system for unified access and acquisition of multiple video data.

As a further improvement of the present invention, the data management platform also includes:

The back-end data storage unit is used to store panoramic fusion pictures, and/or is used to store the video pictures corresponding to each monitoring device according to the time axis as a mark, so that the display interaction system can extract the video data of a single monitoring device or extract each monitoring device at the same time Composed of panoramic fusion screen.

As a further improvement of the present invention, the display interaction system includes:

The display unit is used to display the panoramic fusion picture of the video fusion system data;

The user interaction unit is configured to receive a service control request input by a user, and provide a control function of video fusion display.

The beneficial effects of the present invention are: the multi-channel video collected by shipboard monitoring is fused, breaking the limitation of single camera and lens split screen display in the traditional video monitoring system, organically and seamlessly merging isolated multiple cameras into the panoramic image Among them, the panoramic real-time monitoring of the entire ship is realized. Compared with the viewing of a single camera monitoring image, the fused image improves the efficiency of monitoring viewing and playback monitoring. The multi-channel video fusion display system can perform real-time panoramic fusion of the real-time video collected by the camera to achieve cross-camera tracking and timely target locking.

Description of drawings

Fig. 1 is a structural block diagram of a multi-channel video fusion display system of a ship-borne video surveillance system according to the present invention.

Detailed ways

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments.

As shown in Figure 1, a multi-channel video fusion display system of a shipborne video surveillance system 100 of the present invention includes:

The ship-borne video monitoring system 100 collects video data from multiple locations on the ship through the ship-borne monitoring equipment 101 and encodes them to form video streams, and transmits them to the background data management platform through the communication line 102 for data storage, video stream management, video data display and interact.

The monitoring equipment 101 of the ship-borne video monitoring system 100 includes a ship-specific network video recorder, a monitoring camera, and a monitoring probe. The monitoring device 101 can be installed outside the ship to monitor the situation outside the hull, or it can be the monitoring device 101 installed inside the cabin to monitor the facilities and cargo in the cabin. In order to monitor various positions of the hull, the monitoring equipment 101 located outside or inside the cabin takes pictures of various angles of the hull, thereby forming multiple viewing angles to monitor outside or inside the cabin. However, when these monitoring images are traditionally displayed, they all display images captured by one monitoring device 101 in a grid-like form, which is inconvenient to view as a whole. In the present invention, according to the spatial position of the monitoring device 101 on the hull, corresponding position parameters will be assigned to the monitoring device 101 in turn, and the adjacent parameters will be assigned to the monitoring device 101 that is a spatially adjacent image, such as The monitoring equipment 101 for monitoring the bow assigns a parameter of 1, then assigns a value of 2 for the monitoring equipment 101 for monitoring the plywood adjacent to the bow, etc., which facilitates the synthesis of subsequent monitoring video data and provides a basis for parameters.

The communication line 102 of the shipborne video monitoring system 100 includes the mobile transmission network of the front-end monitoring equipment 101 and the Ethernet or local area network of the background data management platform. The monitoring device 101 can complete the data transmission with the background through the wireless network or the wired network, and transmit the video data to the background data management platform in real time.

The data management platform includes a video fusion system 200 and a display interaction system 300;

The video fusion system 200, by analyzing the feature points of multiple video data, obtains the spatial position and time axis corresponding to each video data, extracts multiple video images of the same time axis, and sequentially fuses them into The panoramic fusion screen of multiple perspectives, the video stream transmits the video frames sequentially according to the sequence of the time axis.

Video fusion system 200 includes:

The feature acquisition module 201 is used to obtain the internal and external parameters of a plurality of monitoring devices 101, extract the corresponding spatial position marks of each monitoring device 101 on the ship in the internal and external parameters, and mark the video pictures formed by the multiple monitoring devices 101 according to the spatial position marks Arranging in order to be spliced and fused; acquiring video data of multiple monitoring devices 101, extracting the time axis of each video data, and using the frame pictures within the same time axis as the same group of video pictures to be spliced and fused.

Before performing image fusion, the feature acquisition module 201 will first extract the video data within the same time axis range as the basis for fusion into the same panoramic image, and then according to the assigned position parameters of the monitoring device 101, the same time axis The video images within the range are sorted, for example, the monitoring video images assigned as 1, 2, 3, and 4 are arranged in sequence, and at this time, the images corresponding to the spatial position of the hull are also captured, which is used for subsequent processing of adjacent images. Preprocessing before fusion.

The analysis and judgment module 202 is used to assign a value to the time axis of each video data by setting a time axis deviation threshold, if the deviation of the time axis assignment of each video data is less than or equal to the time axis deviation threshold, then the time axis of each video data Located within the same time axis range; if the deviation of the time axis assignments of the two video data is greater than the time axis deviation threshold, the video data corresponding to the time axis is discarded.

Before at least two video streams are sent to the fusion module 203, the analysis and judgment module 202 will synchronize the time axis of at least two video streams, and the monitoring device 101 will assign values to the time axis of the code stream when encoding, and the time axis between the code streams The deviation needs to be kept within a certain deviation range. If the deviation threshold is exceeded, the video stream at that moment will be discarded and re-encoded to ensure that the time axis of the video stream generated at the same time is almost at the same time.

The fusion module 203 is used to map the video images arranged in the order of adjacent spatial positions to the panoramic pixel coordinate system through the pre-calibrated panoramic pixel coordinate system for splicing and fusion, and filter the panoramic fusion picture to smooth two adjacent frames Blends brightness between frames.

The panoramic fusion of the fusion module 203 is to combine two or more images with overlapping regions into one large image. The present invention is an image registration method based on feature points, that is, constructing a panoramic pixel mapping matrix between projected images by matching point pairs, completing the mapping between multiple images, and then merging the mapped images to complete panoramic image splicing . The method of the present invention makes full use of a plurality of cameras already installed on the inland ship, and performs panorama splicing and fusion according to the actual situation, which can save equipment resources and reduce costs. At the same time, the splicing effect is tested, and the influence brought by the subjective judgment of the human eye is discarded. When the splicing effect is not good, the splicing calculation is re-stitched, and the splicing parameters are automatically updated to obtain a new panoramic pixel mapping matrix for re-stitching, which can be improved accordingly. Accuracy of stitched images.

Due to the shaking of the ship, small changes such as camera relative position offset may be caused. During the splicing process, the existing all-in-one machine cannot detect in real time the relative position deviation of the camera caused by external factors such as ship shaking, resulting in poor splicing effect. By detecting the splicing effect, the influence of ship shaking can be corrected in time, and the splicing effect can be corrected.

Since there are generally some differences between adjacent photos, after the image stitching is completed, smoothing processing is required to make the entire panoramic stitching image look basically the same in terms of brightness and color. Because it is real-time video splicing, the brightness difference between the front and rear frames of each picture is relatively large. The present invention adopts the form of a filter to smooth the brightness between the current frame and the previous frame, slow down the brightness gradient change, and adjust the entire picture to make its brightness balanced.

The display interaction system 300 displays a panoramic fusion picture of multiple perspectives, and the user calls and consults video data and performs multi-terminal collaborative interaction through management authority. Specifically, the display interaction system 300 includes:

The display unit 301 is used to display the panoramic fusion picture of the video fusion system 200 data;

The user interaction unit 302 is configured to receive a service control request input by a user, and provide a control function of integrated video display.

The display unit 301 can be the display screen of the monitoring room, or a portable mobile terminal screen. The user can send control instructions to the user interaction unit 302 through the virtual keys in the screen or the monitoring console in the monitoring room to call different angles. The panoramic video screen can be adjusted according to the needs, and the shooting angles of each monitoring device 101 can be adjusted to monitor various positions of the hull from a panoramic perspective of multiple angles.

The data management platform also includes:

The front-end data cache unit 401 is used to set multiple cache partitions, correspondingly store the video streams collected by each monitoring device 101, and after summarizing multiple video streams, output them to the video fusion system 200 for unified access and acquisition of multiple video data. The front-end data cache unit 401 pre-buffers the images captured by each monitoring device 101, and transmits the data to the video fusion system 200 in a unified manner, so as to avoid network delays caused by individual monitoring devices 101 when each monitoring device 101 transmits data separately. Partial frame loss occurs when viewing panoramas. And through the way that the video fusion system 200 uniformly accesses the front-end data cache unit 401 to retrieve data, it is ensured that the video stream is stable and the picture is smooth when the panoramic image displayed in the background is operated and viewed.

The back-end data storage unit 402 is used to store the panoramic fusion picture, and/or to store the video picture corresponding to each monitoring device 101 according to the time axis as a mark, so that the display interaction system 300 can extract the video data of a single monitoring device 101 or simultaneously A panoramic fusion picture formed by each monitoring device 101 is extracted.

The historical panoramic video data that has been fused will be transferred to the back-end data storage unit 402 for storage, and the time axis corresponding to each picture will be marked, so that when the video data is subsequently called, a single monitoring device can be smoothly called 101's historical video stream, when calling a panoramic image of a certain time period, the system can directly read the synthesized panoramic video data, or call the video images corresponding to each monitoring device 101 within the same time axis range , quickly combine into the required panoramic video data according to the previously fused rules.

Fusion video playback and historical playback on the same time axis; after the fusion of multiple cameras, unified real-time viewing and unified real-time playback inspection can be performed, which is dozens or even hundreds of times more efficient than traditional single camera screen review. For suspicious people or things, it is easy to find out in advance, give an early warning, and quickly track down after the event, and the safety factor is several times higher than that of the traditional video surveillance system.

The above content is a further detailed description of the present invention in conjunction with specific preferred embodiments, and it cannot be assumed that the specific implementation of the present invention is limited to these descriptions. For those of ordinary skill in the technical field of the present invention, without departing from the concept of the present invention, some simple deduction or replacement can be made, which should be regarded as belonging to the protection scope of the present invention.

Claims (9)

1. The utility model provides a multichannel video fusion display system of on-board video monitor system which characterized in that includes:

the shipborne video monitoring system acquires video data of a plurality of positions of a ship through shipborne monitoring equipment, encodes the video data to form a video stream, and transmits the video stream to the background data management platform through a communication line to perform data storage, video stream management, video data display and interaction;

the data management platform comprises a video fusion system and a display interaction system;

the video fusion system is used for analyzing the characteristic points of a plurality of video data to obtain a spatial position and a time axis corresponding to each video data, extracting a plurality of video pictures of the same time axis, sequentially fusing the video pictures into panoramic fusion pictures with multiple viewing angles according to the sequence of adjacent spatial positions, and sequentially transmitting video frames according to the sequence of the time axes by video streams;

and the display interactive system is used for displaying the panoramic fusion picture of the multiple viewing angles, and the user calls and looks up the video data and carries out multi-terminal collaborative interaction through the management authority.

2. The multi-channel video fusion display system of the shipborne video monitoring system as claimed in claim 1, wherein the monitoring equipment of the shipborne video monitoring system comprises a ship-specific network video recorder, a monitoring camera and a monitoring probe.

3. The multi-channel video fusion display system of the shipborne video monitoring system as claimed in claim 1, wherein the communication line of the shipborne video monitoring system comprises a mobile transmission network of a front-end monitoring device and an ethernet or local area network of a background data management platform.

4. The multi-channel video fusion display system of the shipborne video monitoring system according to claim 1, wherein the video fusion system comprises:

the characteristic acquisition module is used for acquiring internal and external parameters of the plurality of monitoring devices, extracting corresponding spatial position marks of each monitoring device in the internal and external parameters on the ship, and arranging video pictures formed by the plurality of monitoring devices according to the order of the spatial position marks so as to be spliced and fused; the method comprises the steps of obtaining video data of a plurality of monitoring devices, extracting time axes of the video data, and taking frame pictures in the same time axis range as a same group of video pictures to be spliced and fused.

5. The multi-channel video fusion display system of the shipborne video monitoring system according to claim 4, wherein the video fusion system comprises:

the analysis and judgment module is used for assigning values to the time axes of the video data by setting a time axis deviation threshold value, and if the deviation of the time axis assignment of the video data is less than or equal to the time axis deviation threshold value, the time axes of the video data are positioned in the same time axis range; and if the deviation of the time axis assignments of the two video data is greater than the time axis deviation threshold value, discarding the video data corresponding to the time axis.

6. The multi-channel video fusion display system of the shipborne video monitoring system according to claim 1, wherein the video fusion system comprises:

and the fusion module is used for mapping the video pictures sequentially arranged according to the adjacent spatial positions to the panoramic pixel coordinate system for splicing and fusion through the pre-calibrated panoramic pixel coordinate system, filtering the panoramic fusion picture and smoothing the brightness between the two adjacent frames of the fusion picture.

7. The multi-channel video fusion display system of the shipborne video monitoring system according to claim 1, wherein the data management platform further comprises:

and the front-end data cache unit is used for setting a plurality of cache partitions, correspondingly storing the video streams collected by the monitoring equipment, summarizing the video streams, and then uniformly outputting the video streams to the video fusion system to uniformly access and acquire a plurality of video data.

8. The multi-channel video fusion display system of the shipborne video monitoring system according to claim 1, wherein the data management platform further comprises:

and the back-end data storage unit is used for storing the panoramic fusion picture and/or storing the video pictures corresponding to the monitoring equipment according to a time axis as a mark so that the display interactive system extracts the video data of the single monitoring equipment or simultaneously extracts the panoramic fusion picture formed by the monitoring equipment.

9. The multi-channel video fusion display system of the shipborne video monitoring system according to claim 1, wherein the display interaction system comprises:

the display unit is used for displaying a panoramic fusion picture of the video fusion system data;

and the user interaction unit is used for receiving the service control request input by the user and providing a control function of video fusion display.

Images