CN102752600A - Realizing method for reducing time delay of image transmission system - Google Patents

Abstract

The invention discloses an implementation method for reducing the time delay of an image transmission system. The image sending end receives the input image data and divides it into strip data and then outputs it to the coding unit; the coding unit image adopts a parallel mode to receive the strip data, Encoding of strip data and output of encoded strip data; the image sending end adds the encoded strip data to the image receiving end after adding a slice header; the image receiving end extracts the encoded strip data according to the slice header, and encodes The rear strip data is output to the decoding unit; the decoding unit adopts a parallel mode to receive the encoded strip data, decode the encoded strip data and output the strip data; the image receiving end recovers the decoded strip data according to the strip header The obtained strip data is organized into image data and then output. The present invention reduces the time delay of the image transmission system and the implementation resources of the image transmission system through the pipeline processing of the strip data separated from the image.

Description

A Realization Method of Reducing Time Delay of Image Transmission System

technical field

The invention belongs to the field of image transmission data chains, and relates to a realization method for reducing the time delay of an image transmission system.

Background technique

Image transmission system is a technology that integrates image acquisition, processing and transmission. With the extensive development of multimedia technology and wireless transmission technology, image transmission system is widely used in many projects because of its intuitive convenience and rich information. Therefore, it has become a research hotspot. In aerospace engineering applications, especially in image transmission systems, delay is an important index to measure the quality of the system, and the end-to-end delay of the system is usually lower. The lower the delay, the shorter the time the user can watch the decoded image, and use this as a basis to make command judgments. Conversely, if the delay is too high, the real-time performance will be relatively poor, and it will be meaningless for emergency situations, especially for image transmission systems used for precise guidance and control. Therefore, it is necessary to study methods to reduce system delay.

In the image transmission system, the delay mainly includes several parts: transmission delay of image input and output, image encoding and decoding processing delay, buffer delay and channel encoding and decoding processing delay. It can be seen that the delay is mainly generated at the image codec end. Therefore, the reduction of system delay focuses on reducing the delay of image codec.

The currently published methods for reducing the time delay at the image encoding and decoding end are mainly divided into two categories. The first category is to study the rate control algorithm used in image encoding. By trying to make the encoded data of each frame close to the target rate, reduce The method of buffering data at the encoding end achieves the purpose of reducing the buffering delay, but this strategy can only achieve the stability of the encoded data within a certain period of time, and cannot be completely accurate enough that each frame just reaches the target bit rate, and the algorithm is usually more complicated and requires a lot of calculation. The second category is to study the fast algorithm of image encoding and decoding, so as to reduce the processing delay of image encoding and decoding, but the improvement of speed will inevitably bring a certain loss of quality.

Contents of the invention

The problem solved by the technology of the present invention is to overcome the deficiencies of the prior art and provide a method for reducing the time delay of the image transmission system. Delay and implementation resources.

Technical solution of the present invention is:

An implementation method for reducing the time delay of an image transmission system, the image transmission system includes an image sending end and an image receiving end, wherein the image sending end includes a coding unit; the image receiving end includes a decoding unit, and the image sending end and the image receiving end are based on The following steps carry out image data transmission:

The image sending end receives the input image data and divides it into strip data and outputs it to the coding unit;

The encoding unit image adopts parallel processing to receive the strip data, encode the strip data and output the encoded strip data;

The image sending end adds the encoded strip data to the image receiving end after adding a strip header;

The image receiving end extracts the encoded strip data according to the slice header, and outputs the encoded strip data to the decoding unit;

The decoding unit adopts parallel processing to receive the encoded strip data, decode the encoded strip data and output the strip data;

The image receiving end organizes the decoded and restored slice data into image data according to the slice header, and then outputs it.

Further, the encoding unit uses at least two buffer areas with the same size as the stripe data to buffer and encode the input stripe data.

Further, the parallel processing method of the encoding unit is: while encoding the slice data at the current moment, output the encoded slice data at the previous moment, and output the encoded slice data at the next moment data is received.

Further, the slice header includes a slice flag header and a slice number, and the image receiving end determines a complete coded slice data according to the slice flag header, and organizes the decoded and restored slice data according to the slice into image data.

Further, the decoding unit uses at least two buffer areas with the same size as the encoded strip data to buffer and encode the input encoded strip data.

Further, the parallel processing method of the decoding unit is: while decoding the encoded slice data at the current moment, output the decoded slice data at the previous moment, and output the encoded slice data arriving at the next moment Stripe data is received.

Compared with the prior art, the present invention has the following advantages:

In the invention, the image sending end divides the image data into a plurality of strips, takes the strip data as a unit, and uses a parallel processing method to overlap the processing time of multiple parts, so as to achieve the purpose of reducing system delay. The method is applicable to video compression algorithms for image processing by blocks (that is, the strips described in the present invention are sets of blocks), such as H.264 algorithm and MPEG4 algorithm, and is easy to implement in engineering without loss of image quality.

The processing mode of the image encoding end of the existing image transmission system is to start the compression encoding process after receiving a whole frame of image. However, the present invention uses strip data as a unit to perform image input and compression. After inputting a strip data, the compression starts, and the parallel processing of image input and encoding is started in a single frame image, and the image input delay and image compression are realized. The overlapping of compressed encoding delay effectively reduces the system delay caused by the encoding end.

The image receiving end of the existing image transmission system starts the decoding and decompression process after receiving the code stream data of a whole frame of image, while the code stream structure of the present invention organizes the coded strip data as a unit, so the image receiving end After receiving an encoded strip data, decoding and decompression can be started, which realizes the overlap of code stream input buffer delay and image decoding and decompression delay, effectively reducing the system delay caused by the decoding end.

Since the present invention processes strip data as a basic unit, most of the consumed memory is allocated using strips as a basic unit. For example, for the present invention, the original image input buffer at the image encoding end can be allocated at least 2 slice sizes, while for the prior art, the original image input buffer needs to be allocated at least 2 full frame image sizes, which is similar to image decoding end. Therefore, the present invention has the characteristics of consuming less memory resources and saving memory space.

The invention has no specific requirements on the complexity of the input image, and can achieve the purpose of reducing the time delay of the image transmission system no matter it is a complex image with large dynamics or a simple image with small dynamics, and has wide adaptability. Moreover, the algorithm is simple to realize, has high real-time performance, and is easy to implement in engineering.

Description of drawings

Fig. 1 is a flowchart of the present invention

FIG. 2 is a schematic diagram of parallel processing at the sending end;

Fig. 3 is a schematic diagram of parallel processing at the receiving end;

Fig. 4 is a schematic diagram of the process flow of the image sending end of the present invention;

Fig. 5 is a schematic flow diagram of the image receiving end of the present invention.

Detailed ways

Flow chart shown in Figure 1, the present invention comprises the following steps:

(1) The original digital image digitized by A/D sampling is first sent to the FIFO built by the FPGA as the image acquisition unit of the image sending end for buffering, and the FPGA collects a strip of data and sends it to the encoding unit of the image sending end The DSP sends an interrupt signal, and after receiving the interrupt signal, the DSP triggers the EDMA background operation to quickly receive a strip of image data sent by the FPGA.

(2) The encoding unit DSP immediately starts the image encoding operation after receiving a slice of data, encodes the received slice of data, and uses EDMA to output the coded stream data of the slice. The parallel processing method of the encoding unit is shown in Figure 2. One frame of image data is equally divided into n strip data, where: S _mk represents the input time of one strip data, m is the image frame number, k is the strip number, and the strip The band number range is (1, n); C _mk represents the encoding time of one slice; T _mk represents the code stream data transmission time of one slice.

In Figure 2, due to the fact that the complexity of the data in each strip is different, the encoding time of each strip and the length of the code stream data are inconsistent, but the transmission rate is constant, so the transmission time of the code stream data in each strip is also It is not the same, and this transmission also involves the process of rate control and data buffering, so the actual working process cannot be accurately represented by Figure 2. Figure 2 simply describes a process of parallel processing by the sender according to stripes. All the work at the image sending end begins encoding and transmission after inputting a slice, which means that all the work at the image sending end is advanced by n-1 slices, thereby achieving the purpose of reducing the delay at the image sending end.

(3) The encoding unit encodes the data in slices, repackages the stream data of a slice every time it is generated, and adds slice header information before the code stream data of the slices. As shown in Table 1, the slice header and the slice number are used to identify and distinguish a complete slice code stream data, and the slice encoding mode and slice quantization value are used by the decoding unit at the image receiving end.

Table 1

illustrate Stripe Logo Header stripe number strip coding method strip quantization value length 32bit 6bit 1bit Between 1bit and 7bit

(4) The DSP, which is the decoding unit of the image receiving end, performs the receiving and moving operation of the code stream in the background through EDMA. Every time a certain amount of code stream data is received, EDMA triggers an interrupt to notify the CPU of the DSP, and the CPU performs subsequent processing on the received code stream data. At the same time, the EDMA continues to receive the code stream data, and uses the EDMA to output the strip recovery data.

(5) The decoding unit DSP searches for a complete strip code stream data while receiving the code stream data. The search principle is: search for the slice flag header in the slice header, once a slice flag header is found, continue to search for the next slice flag header, after two slice flag headers are found, it means that a slice flag header has been received For the complete strip code stream data, the actual strip code stream data is extracted and decoded according to the slice header information, and the strip recovery data generated by decoding is output. The parallel processing method of the decoding unit is shown in Figure 3. Among them: T _mk represents the input time of the code stream data of a strip, m is the image frame number, k is the strip number, and the range of the strip number is (1, n); D _mk represents the decoding time of a code stream data of a strip ; R _mk represents the output time of a strip recovery data.

Similar to the analysis of step (2), in reality, the data lengths of each strip code stream are different, so the transmission time and decoding time are also different, so the actual working process cannot be accurately represented by Figure 3. Figure 3 simply describes the process of parallel processing by the image receiving end by slices. Explain all the work of the image receiving end. Since the code stream data is organized in units of the strip code stream data, the decoding operation can be started after receiving a complete strip code stream data, thereby reducing the time spent on the image receiving end. extended purpose.

(6) For the recovered strip data, store it in the decoding image buffer of the image receiving end according to the strip number, and judge whether the strip is the last strip according to the strip number, if so, decode the whole frame The image recovery is completed and can be used for subsequent processing such as display. This is because the image sending end encodes and transmits the original slice data sequentially according to the slice numbers, so the image receiving end receives and decodes the restored slice data sequentially. Therefore, when the slice data whose slice number is the last slice is to be recovered, it also means that a whole frame of decoded images is recovered.

Example:

The experimental conditions of this embodiment are:

(1) The input is a digital video image with a size of 720x576x8bit, and the frame rate is 25f/s;

(2) The coding output code rate is 2Mbps;

(3) The stripe data resolution means the stripe data size is 720x16x8bit.

Combined with the above-mentioned test conditions, according to the schematic flow chart of Figure 4 and Figure 5 .

image sender

(1) FPGA caches the received digital image data, and when the cache data reaches 720x16x8bit size, an interrupt signal is given to DSP to trigger EDMA to move the input image data. The EDMA transfer rate is very high, and the transfer delay caused by it is negligible. According to the frame rate, the time for the FPGA to receive and buffer a strip of data is:

(2) DSP notifies the encoding main task after receiving a complete strip data, and starts encoding processing, and the strip code stream data generated by encoding is stored in the code stream sending buffer to be sent. Therefore, when the DSP background EDMA moves other strip data, the DSP can encode the strip data; while receiving and encoding the strip data, another channel of the DSP background EDMA will generate and organize the strip data. The code stream data is output; thus the parallel processing of strip data input, encoding and output is realized. Compared with the prior art method of buffering one frame of image and then starting encoding, the time saved can be roughly calculated as:

(3) Organize the code stream data in the code stream structure with stripes as the unit, and add the key information of the stripe header according to Table 1, where the stripe header is 0x00000001, the stripe number ranges from 1 to 36, and the stripe code The mode is 0 (I frame) or 1 (P frame), and the slice quantization value range is (0, 51). After stream organization, output it to the image receiving end at a rate of 2Mbps.

的时间，有效降低了图像发送端的时延。According to the above analysis, it can be seen that under the experimental conditions, the present invention can reduce

time, effectively reducing the delay of the image sending end.

image receiver

(1) The DSP at the image receiving end uses EDMA to receive the continuous serial code stream data sent by the image sending end at a rate of 2M. After receiving 120 words of data, it informs the CPU of the DSP through a completion interrupt. The DSP searches the code stream data, and if two 0x00000001 strip flag headers are found, it means that a data stream containing a complete strip code stream data is retrieved, and then decodes it.

(2) Perform decoding processing on the slice code stream data according to the slice encoding method and the slice quantization value parameter in the slice header, and recover a slice decoded data. While performing the above processing on the strip code stream data composed of encoded strip data, EDMA still receives the code stream data in the background; at the same time, use another channel of EDMA to process the generated strip recovery data in the background output. Therefore, the parallel processing of code stream data reception, coded strip data decoding and strip data output is realized. Assume that the code stream data length of a frame of image is: 2000000/25=80000 bits, and the size of one strip code stream data is: 80000/36=2222 bits, compared with the prior art receiving a whole frame of image code stream data The method of starting the decoding process again, the time saved can be roughly calculated as:

(3) Store the recovered strip data in the corresponding position of the decoded image buffer according to the strip number. If the strip number is 36, it means that the last strip data of the frame image has been recovered, that is, a The complete decoded image can be displayed and processed.

According to the above analysis, it can be seen that under the experimental conditions, the present invention can reduce the time of 38.89 ms by rough calculation compared with the prior art, effectively reducing the time delay of the image receiving end.

The parts not described in detail in the present invention belong to the well-known technology in the art.

Claims (6)

1. implementation method that reduces the image delivering system time delay, said image delivering system comprises image transmitting terminal and image receiving terminal, wherein, the image transmitting terminal comprises coding unit; The image receiving terminal comprises decoding unit,

It is characterized in that image transmitting terminal and image receiving terminal carry out image data transmission according to following steps:

The image transmitting terminal outputs to coding unit after receiving the view data of input and being divided into strip data;

The coding unit image adopts the mode of parallel processing to carry out the reception of strip data, the output of the coding of strip data and coding back strip data;

The image transmitting terminal sends to the image receiving terminal after coding back strip data is added slice header;

The image receiving terminal extracts coding back strip data according to slice header, and the back strip data of will encoding outputs to decoding unit;

The reception of strip data after said decoding unit adopts the mode of parallel processing to encode, the decoding of coding back strip data and the output of strip data;

After being organized into view data, the strip data that the image receiving terminal will recover after will decoding according to slice header exports.

2. a kind of implementation method that reduces the image delivering system time delay as claimed in claim 1 is characterized in that: said coding unit adopts at least two buffer areas identical with the strip data size that the strip data of input is carried out buffer memory and coding.

3. according to claim 1 or claim 2 a kind of implementation method that reduces the image delivering system time delay; It is characterized in that: the parallel processing mode of said coding unit is: when the strip data of current time is encoded; Strip data is exported behind the coding that the coding of previous moment is accomplished, and the strip data that next arrives is constantly received.

4. a kind of implementation method that reduces the image delivering system time delay as claimed in claim 1; It is characterized in that: said slice header comprises band sign head, bar reel number; The image receiving terminal is confirmed strip data behind the complete coding according to band sign head, and is organized into view data according to the strip data that will recover after will decoding behind the band.

5. a kind of implementation method that reduces the image delivering system time delay as claimed in claim 1 is characterized in that: strip data was carried out buffer memory and coding after said decoding unit adopted at least two buffer areas identical with coding back strip data size to the coding of input.

6. like claim 1 or 5 described a kind of implementation methods that reduce the image delivering system time delay; It is characterized in that: the parallel processing mode of said decoding unit is: when strip data is decoded behind the coding of current time; Strip data to previous moment decoding back produces is exported, and strip data behind next coding that arrives is constantly received.

Images