CN103248964B - Based on the Vehicular video transmission system of RTP/RTCP - Google Patents

Abstract

A vehicle-mounted video transmission system based on RTP/RTCP is characterized in that the entire vehicle-mounted video transmission system of the present invention is jointly constructed by a network transmission control module and some N movable clients, and each vehicle-mounted terminal forms a VANET network, The client is a vehicle-mounted terminal placed in the vehicle, and each vehicle-mounted terminal is equipped with Intel? IPP library, the vehicle video transmission system is client/server mode. The present invention designs a vehicle-mounted video transmission system based on RTP/RTCP, considers that different video compression parameters will affect the final transmission quality, dynamically adjusts the video compression quality in real time according to the packet loss rate detected by the system, and the system of the present invention can dynamically Adjust video transmission quality.

Description

Car Video Transmission System Based on RTP/RTCP

technical field

The invention relates to a video transmission system in the technical field of vehicle wireless network communication.

Background technique

With the development of transportation and information industry, various video applications based on vehicle networks have attracted widespread attention. Vehicle safety assisted driving systems, unmanned vehicle systems, real-time road condition monitoring systems beyond the line of sight, real-time audio and video communication systems, and multimedia entertainment information sharing systems play an important role in ensuring traffic safety, improving traffic efficiency, and enriching people's lives. meaning. The construction of these systems will inevitably involve the encoding and transmission of video on the vehicle network. Video technology has successfully commercialized applications such as video conferencing and video-on-demand in wired and wireless networks. How to achieve smooth video transmission and video sharing between vehicles in the vehicle wireless network communication is a hot spot in the field of vehicle wireless network research.

First, vehicle network communication is extremely unstable. The reflection of the signal by the object, the high-speed movement characteristics of the vehicle network, the complexity of the road structure and the roadside environment, etc., make the channel characteristics of the vehicle self-organizing network different from the general static or low-speed Adhoc network. The quality of data transmission between nodes in a vehicle ad hoc network depends on specific transmission scenarios. In view of the up-and-down communication characteristics of vehicular ad hoc networks, if real-time video communication is to be performed, the quality of the obtained video must vary with the channel. If the video quality output by the video compression module is good, when the channel goes bad, channel congestion tends to occur easily. On the contrary, the output video quality of video compression is poor, which is suitable for unstable or poor channel conditions, but it cannot make full use of the bandwidth resources when the channel is good.

Second, video frames also have their own unique characteristics. Video compression is in units of GOP. GOP contains different frame types: I frame, P frame and B frame. The losses caused by the loss of different types of frames are not the same. The loss of an I frame will affect the correct decoding in the entire GOP. In order to minimize the failure of decoding caused by lost I frames, the usual solution is to reduce the length of the GOP. However, as the GOP length decreases, the number of I frames increases, the efficiency of compression decreases, and the generated code rate increases, resulting in an increase in the amount of data. As the amount of data increases, the corresponding UDP transmission packets also increase. If network congestion occurs, the number of packet loss will potentially increase. Therefore, the packet loss rate and the length of the GOP are a "chicken-egg" paradox. If the packet loss rate is used as a parameter to measure the stability of the network transmission channel, and the GOP size and video bit rate or other compression parameters are actively adjusted, a relatively ideal transmission effect can be achieved.

The following are notes on some terms in this manual:

Adhoc network (peer-to-peer multi-hop mobile network capable of temporary fast automatic networking)

GOP (GroupOfPicture, picture group, a GOP is a group of continuous pictures)

UDP (the abbreviation of UserDatagramProtocol is not a connection protocol, so it has the advantages of low resource consumption and fast processing speed, so usually audio, video and ordinary data use UDP more when transmitting, because even if they occasionally lose one or two data packets , and will not have much impact on the receiving results. The main function of the UDP protocol is to compress network data traffic into the form of data packets.)

DirectShow (DirectShow is an open development environment, you can customize your own components as needed)

VANET (VehicularAd-hocNETwork, vehicle ad hoc network)

IntelIPP library ("Intel Integrated Performance Primitives" is a set of cross-platform software function libraries that provide a wide range of multimedia functions, such as audio decoders, image processing, signal processing, voice compression and encryption mechanisms)

NS2 (NS2 is a simulation software, a tool that can simulate communication network protocols)

The above terminology itself is already known in the art, so the related content should not be regarded as insufficient disclosure.

Contents of the invention

The invention provides a vehicle-mounted video transmission system based on RTP/RTCP. By dynamically adjusting the video compression quality in real time according to the packet loss rate detected by the system, the system of the invention can dynamically adjust the video transmission quality.

In order to achieve the above object, the system technical solution of the present invention is characterized as:

A vehicle-mounted video transmission system based on RTP/RTCP is characterized in that the entire vehicle-mounted video transmission system of the present invention is jointly constructed by a network transmission control module and some N movable clients, and each vehicle-mounted terminal forms a VANET network, The client is a vehicle-mounted terminal that can be placed in a vehicle, and each vehicle-mounted terminal is equipped with an IntelIPP library, and the vehicle-mounted video transmission system is a client/server mode. Each vehicle-mounted terminal comprises 5 functional sub-modules altogether, and they are video acquisition module, video codec module, information processing module, video playback module, file storage module, and these functional sub-modules are all constructed on the described IntelIPP storehouse, wherein: The video acquisition module is used for the acquisition of video data, obtains video original data from sensor hardware equipment, or obtains video original data from file video streams, etc., and the video acquisition module is respectively connected with a video playback module and a video encoding module for local display Encoding and compression of video files or before transmission;

The video codec module is used to reduce transmission video data redundancy;

The information processing module is to complete the preprocessing and video compression operation of the data of the video acquisition module;

The video playing module is connected with the video acquisition module and the video decoding module respectively, and is used to play and display the video data data collected locally or the video data decoded from other mobile vehicle-mounted terminals transmitted from the VANET network;

The file storage module is used for storing locally collected video data and received video data from other vehicle-mounted terminals, and then used to send and transmit video files to other receiving terminals.

The network transmission control module of the present invention is constructed on the VANET, and the network transmission control module is used for the video data transmission between the mobile vehicle-mounted terminal and the vehicle-mounted terminal. The network transmission control module includes a software module for dynamically adjusting the video transmission quality algorithm, which is characterized as: the software module dynamically adjusts the video transmission quality based on the network packet loss rate and the complexity of the video series, and evaluates the video after it is transmitted through the simulated vehicle network. According to the video quality of different complexities, the factors affecting the quality of video transmission GOP, compression and quantization parameters qp, bitrate, and packet error rate are tested, and the best parameter combination is obtained.

Described dynamic adjustment video transmission quality algorithm software module, the method for its implementation comprises the following steps,

First, the acquisition step of the network packet loss rate is used to obtain the current network packet loss rate;

Then, the video sequence complexity judging step is used to distinguish the motion intensity of the video sequence;

Finally, the dynamic quality adjustment step is used to divide the network quality into three levels according to the network packet loss rate: good, bad, and very poor. Before encoding the frame, if the network quality changes from one state to another, according to For the video sequence, adjust the encoding parameters to meet the optimal compression parameters under the network conditions.

By designing a vehicle-mounted video transmission system based on RTP/RTCP, the present invention considers that different video compression parameters will affect the final transmission quality, dynamically adjusts the video compression quality in real time according to the packet loss rate detected by the system, and achieves the improvement of the final transmission quality the goal of. Innovation points and beneficial effects:

(1) Design a complete vehicle network video transmission scheme and system. By making full use of the excellent compression performance of the H.264 encoder and the powerful computing power of the dual-core processor, the video collection, encoding, decoding, and transmission functions of the vehicle system are realized.

(2) According to the packet loss rate of the vehicle network, it provides optimized compression parameter configuration for different network conditions, and can dynamically adjust the video transmission quality. It effectively overcomes the disadvantages of not being able to make full use of the bandwidth when the network is dynamically changing under a single compression parameter or that the network is congested due to excessive data volume.

Description of drawings

Fig. 1 is a functional block diagram of the entire vehicle-mounted video transmission system of the present invention.

Fig. 2 is a hierarchical diagram of the protocol stack of the present invention.

Figure 3 Video capture flow chart.

Figure 4 Flowchart of the encoding module.

Figure 5 is a flowchart of the decoding module.

Figure 6 is a structural schematic diagram of the EC5-1719CLDNA development board and external interfaces of the hardware part of the vehicle terminal.

Fig. 7 is a schematic diagram of the data flow of the vehicle terminal in the software part of the present invention.

Figure 8 is a video transmission link.

Fig. 9 is a flowchart of an adaptive video quality algorithm.

detailed description

The technical solution of the present invention will be further introduced below in conjunction with the accompanying drawings.

As shown in Figure 1, the network transmission control module and some N movable clients jointly construct the entire vehicle-mounted video transmission system of the present invention, and each vehicle-mounted terminal forms a VANET network, and the client is a vehicle-mounted terminal placed in the vehicle , each of the vehicle-mounted terminals is installed with an IntelIPP library, and the vehicle-mounted video transmission system is a client/server mode. Each vehicle-mounted terminal comprises 5 functional sub-modules altogether, and they are video acquisition module, video codec module, information processing module, video playback module, file storage module, and these functional sub-modules are all constructed on the described IntelIPP storehouse, wherein: The video acquisition module is used for the acquisition of video data, obtains video original data from sensor hardware equipment, or obtains video original data from file video streams, etc., and the video acquisition module is respectively connected with a video playback module and a video encoding module for local display Video files or encoding compression before transmission. As shown in Figure 3, the collection process is developed by DirectShow, and the realization of video collection is not a part of the contribution of the technical solution of the present invention to the prior art.

The video codec module is used to reduce the redundancy of transmitted video data, so as to solve the problem that the video data volume is too large and the VANET network bandwidth cannot be satisfied. The video codec module is built on the IntelIPP library, including two parts: the encoding module and the decoding module.

The basic process of coding module work is shown in 4. The video coding module is based on the development tools IPP and UMC provided by Intel Corporation. H.264 encoding and compression is performed on the cached data in the data queue collected by SampleGrabber to fit the current limited VANET network bandwidth.

The flow of the decoding module is shown in Figure 5. After receiving the frame data from the VANET network, the user extracts the image size, color format, encoder type and other information, initializes the decoding environment, and calls the decoding function to obtain the decoded frame, and finally uses the VideoRenderer provided by UMC to display the image for video playback module to play.

The above encoding and decoding processes belong to the prior art, and the encoding and decoding technology itself is not a contribution part of the technical solution of the present invention to the prior art.

The information processing module is to complete the preprocessing of the data of the video acquisition module and the video compression operation.

The video playback module is respectively connected with the video acquisition module and the video decoding module, and is used to play and display locally collected video data or decoded video data transmitted from the other party's (other) mobile vehicle terminal transmitted from the VANET network.

The file storage module is used to store locally collected video data and video data received from the counterpart (other) vehicle-mounted terminal, and then used to send and transmit video files to other receiving terminals.

The network transmission control module is built on VANET, and the network transmission control module is used for video data transmission between the mobile vehicle-mounted terminal and the vehicle-mounted terminal, and supports unicast and multicast modes. The transport layer of the network transmission control module adopts the RTP/RTCP real-time transmission and control protocol to realize the data feedback and control on the two transmission modes of the VANET network. The video transmission function of this module is realized by the RTPMulticasting multicast rooted model. All clients join the same multicast address and bind the same local UDP port number A to receive multicast data. The server side (that is, a certain vehicle terminal) is bound to another port number B for data exchange with all clients. In this way, when client A sends data to server port number B, the server sends data to the multicast address and port A after receiving it, so that all clients can receive the video data transmitted by client A.

In addition, the network transmission control module includes an algorithm software module for dynamically adjusting the video transmission quality, which is characterized as: the software module dynamically adjusts the video transmission quality based on the network packet loss rate and the complexity of the video series, and evaluates the quality of the video in the simulated vehicle. The video quality after network transmission is tested for video sequences of different complexity on the factors affecting video transmission quality GOP, compression quantization parameter qp, bitrate, and packet error rate, and the best parameter combination is obtained.

The method implemented by the software module is characterized in that it comprises the following steps,

First, the acquisition step of the network packet loss rate is used to obtain the current network packet loss rate;

Then, the video sequence complexity judging step is used to distinguish the motion intensity of the video sequence;

Finally, the dynamic quality adjustment step is used to divide the network quality into three levels according to the network packet loss rate: good, bad, and very poor. Before encoding the frame, if the network quality changes from one state to another, according to For the video sequence, adjust the encoding parameters to meet the optimal compression parameters under the network conditions.

Specific examples are given below to further introduce the technical solution of the present invention.

The present invention is based on the vehicle video transmission system of RTP/RTCP and realizes steps as follows:

(1) Vehicle terminal hardware architecture

Each vehicle-mounted terminal should have strong processing capabilities to meet the needs of video compression operations and sensor data processing of the video acquisition module. In this embodiment, the EC5-1719CLDNA development board produced by EVOC is selected. The EC5-1719CLDNA is a high-performance single-board computer designed with an Intel notebook computer chipset 945GM. The processor is IntelCore2Duo, and the bandwidth between the north and south bridges reaches 10Gb. /s Direct Media Interface DMI. It is characterized by high performance, low power consumption and rich expansion card interfaces. Based on the development board, memory, storage, display LCD, touch screen, camera, wireless network card, microphone and other devices can be expanded as needed, as shown in Figure 6.

The video capture module acquires video data from hardware devices, which include capture cards, TV receiver cards, cameras, video recorders, etc.

(2) Vehicle terminal software architecture

The vehicle-mounted terminal belongs to the common C/S framework, that is, it consists of two parts: the video sending end and the video receiving end. The difference is that each vehicle-mounted terminal node of the present invention belongs to a peer-to-peer node, and can be used as a video sender or as a video receiver. Each vehicle-mounted terminal system has modules to collect and encode videos through cameras, etc. There are also modules (threads) that are used to receive real-time video from the other party's vehicle terminal and play it. According to the functions, the vehicle terminal can be divided into video acquisition module, video encoding module, network transmission control module, video decoding module, and video playback module, as shown in Figure 7.

The present invention adopts DirectShow to realize video collection.

The network transmission control module is built on VANET. The transport layer of the network transmission control module adopts the RTP/RTCP real-time transmission and control protocol to realize the data feedback and control on the two transmission models of the VANET network.

The video codec module is built on the IntelIPP library, and video coding and decoding can be realized based on Intel's IPP codec library.

The video transmission of the entire vehicle terminal can form a link as shown in Figure 8 according to the flow direction of the video data and the technology adopted.

The method steps of the present invention dynamically adjust video transmission quality based on network packet loss rate and video series complexity are as follows:

(1) Simulation platform construction and parameter combination acquisition

Based on the two tools NS2 and Evalvid, the Intel IPPH.264 encoder extension is used to evaluate the video quality after the video is transmitted through the simulated vehicle network. The vehicle network channel error is simulated by using the Gilbert-Elliott error model. For video sequences of different complexity, the factors affecting video transmission quality GOP, compression and quantization parameters qp, bitrate, and packet error rate are tested to obtain the best parameter combination.

(2) Adaptive video quality algorithm process

1) Divide network channels into three types: GOOD, BAD, WORSE.

When the packet loss rate is lower than 1%, define the channel status as GOOD;

When the packet loss rate is greater than 1% and less than 10%, define the channel state as BAD;

When the packet loss rate is greater than 10%, define the channel state as WORSE.

The channel conditions of the vehicular network are constantly switching between these three channel types as the environment changes.

2) every once in a while , to calculate the packet loss rate.

If the channel packet loss rate is lower than 1%, provide high-quality video parameter combinations according to three sequence complexities, as shown in Table 1;

If the packet loss rate satisfies [1%-10%), IPPPP sequences are provided according to three sequence complexities, GOP=5, as shown in Table 2;

Assuming that the packet loss rate reaches 10% or even higher, it indicates that the vehicle is damaged due to factors such as excessive distance, or to a dead end of the road. Therefore, at this time, the GOP combination of IBBBP is used, as shown in Table 3, and the code rate is reduced at the same time. At this time, all B frames will not affect the decoding of successfully transmitted frames after packet loss. Compared with the previous combinations, the comprehensive Video quality will improve.

The three parameter combinations for reference are shown in Tables 1, 2, and 3, respectively. The detailed algorithm flow is shown in Figure 9.

Table 1 Combination of video compression parameters with packet loss rate less than 1%

Table 2 Combinations of video compression parameters with a packet loss rate of 1%-10%

Table 3 Combination of video compression parameters with packet loss rate greater than 10%

Claims (1)

1. the Vehicular video transmission system based on RTP/RTCP, it is characterized in that, jointly built by Internet Transmission control module and several transportable clients, each car-mounted terminal forms VANET network, described client is the car-mounted terminal be placed in car, each car-mounted terminal is provided with IntelIPP storehouse, and described Vehicular video transmission system is Client/Server;

Each car-mounted terminal comprises 5 function sub-modules altogether, they are video acquisition module, coding and decoding video module, message processing module, video playback module, file storage module, these function sub-modules are all structured on described IntelIPP storehouse, wherein: described video acquisition module is used for the collection of video data, video original data is obtained from sensor hardware equipment, or obtain video original data from file video flowing etc., video acquisition module respectively with video playback module, video encoding module connects, for the compression coding before local display video file or transmission,

Described coding and decoding video module is for reducing transmitting video data redundancy;

Described message processing module, for completing preliminary treatment to video acquisition module data and video compression computing;

Described video playback module is connected with video acquisition module and Video decoding module respectively, for playing the local video data gathered of display or the decoded video data of other mobile car-mounted terminal come from VANET Internet Transmission;

Described file storage module, for storing the local video data gathered and the video data come from the reception of other car-mounted terminals, is used further to send transmitting video files to other receiving terminals;

Described Internet Transmission control module, is structured on VANET, and Internet Transmission control module is used for the video data transmission between movable car mounted terminal and car-mounted terminal; This Internet Transmission control module comprises dynamic regulating video transmission quality algorithm software module, be characterized by: this software module is that packet loss Network Based and video sequence complexity carry out dynamic regulating video transmission quality, the video quality of assessment video after transmitting through simulation vehicle network out, for the video sequence of different complexity on affecting the factor GOP of video transmission quality, compression quantization parameter qp, code check bitrate, packet error rate test, show that optimal parameter combines;

Described dynamic regulating video transmission quality algorithm software module, its method implemented, comprises the following steps,

First, the obtaining step of network packet loss rate, for obtaining current network packet loss;

Then, video sequence complexity determining step, for distinguishing the motion intense degree of video sequence;

Finally, dynamic conditioning quality step, for according to network packet loss rate, network quality is divided into Three Estate: good, poor, very poor, before coding present frame, if network quality is from a state variation to another state, according to the sequence of video, adjustment coding parameter, to meet the optimal compression parameter under this network condition.