Method based on dynamic interleaving in the video transmission of slice level
Technical field
The present invention is a kind of application process that is used to improve the Internet video data transmission credibility, belongs to reliability transmission technical field in the network.
Background technology
AVI (Audio Video Interleaved) is the Audio Video Interleaved form.Be with voice and image sync file format together.It has adopted a kind of lossy compression method mode to video file, and compression ratio is higher, although therefore face face quality is not very good, but its range of application is still very extensive.AVI supports 256 looks and RLE compression.AVI information is mainly used on the multimedia CD, is used for preserving various image informations such as TV, film.AVI does not have MPEG so complicated, and from the WIN3.1 epoch, it has just emerged.Its most directly advantage be exactly compatible good, call convenient and image quality good, therefore also usually with DVD mutually and title.But its shortcoming also is fairly obvious: volume is big.Also be that we have just seen the birth of MPEG-1 and MPEG-4 because of this point.Under the much the same situation of MPEG-2 formatted file volume; The video quality of AVI form will differ from much comparatively speaking; Present DTV and the success of DVD are on the Moving Picture Experts Group-2 that was based upon before more than ten years, and it is very practical that this standard once was proved to be, but it seems now somewhat out-of-date.With more efficiently, more practical, and the technology of can good use stronger disposal ability replace MPEG-2 trend clearly; The arguement that at present which kind of technology will replace MPEG-2 for is also constantly continuing, and that wherein relatively have an optimistic view of is international standard MPEG-4 and H.264.
H.264 the standard compression system is by video coding layer (VCL) and network abstraction layer (Network AbstractionLayer, NAL) two parts composition.Comprise VCL encoder and VCL decoder among the VCL, major function is video data compaction coding and decoding, and it comprises compression unit such as motion compensation, transition coding, entropy coding.NAL then is used to VCL a unified interface that has nothing to do with network is provided; It transmits it after being responsible for video data encapsulated packing in network; It adopts uniform data format, comprises video data and framing, logic channel signaling, timing information, sequence conclusion signal of the header packet information of single byte, a plurality of bytes etc.Comprise storage mark and type code in the packet header.Storage mark is used to indicate current data not belong to by the frame of reference.Type code is used for the indicating image type of data.VCL can transmit the coding parameter by current network condition adjustment.
Although the basic structure of H.264 encoding with H.261, H.263 be similarly, it improves in a lot of links: 1, multiple better estimation.2, more accurate infra-frame prediction.3, unified VLC.Because H.264 the advantage that H.264 aspect compression performance, is had will be applied in fields such as real-time video communication, radio and television, video storage broadcast.
There are coded video sequence, single image, section (Slice), macro block, piece in the unit that in H.264, possibly be selected as parallel basic task.But in practical application, have only coded video sequence, Slice and macro block really to be used as basic task and moved by parallel encoding.The coded video sequence conduct is coding unit independently, is carrying out having sizable advantage in the parallel encoding processing.But its shortcoming also is that significantly parallel fineness ratio is bigger, and the unbalanced situation of load takes place between each processor easily.When the user interactions demand takes place, the scheduling difficulty.The occasion that often changes for demand is unaccommodated; If with single image as the operation of basic parallel task coding, so under the normal condition, follow-up image in the coded video sequence, must have behind the previous image coding data as a reference, could accomplish the work of encoding.If the required reference data of image to be encoded is encoded do not finish, just must wait for.The mutual wait that this has not only caused between the task has increased the traffic between the processor, also is difficult to the management of dispatching, and its scramble time also can be higher than the operation result of uniprocessor sometimes.Therefore, based on the parallel encoding mode of image, lost the meaning of initial parallel encoding; On the granularity thinner, carry out the parallel processing of video coding than macro block; To cause a large amount of inter-processor communications; Because the estimation scheduling algorithm need be searched in the zone outside local storage, obviously take thinner division methods not only can not improve the performance of system, can cause the serious decline of systematic function on the contrary; Between the macro block among the same Slice, still have bigger dependence.When coding, can cause the communication between processor equally, further possibly cause the mutual wait between the processor.
The reliability that in video transmission, adopts the data interlacing Methods for Coding to improve transmission is not any fresh thing; Data interleaver commonly used at present has: regular interleaver, irregular interleaver, random interleaver; Wherein regular data interleaver and irregular data interleaver realize that simply algorithm complex is lower, and the decentralization of random interleaver generation data is higher; Can solve the burst packet loss to greatest extent, amount of calculation is bigger.For data interleaver, the sphere of action of its interleaving data directly influences the quality of transfer of data, and when the data interlacing degree that is acted on is bigger than normal, the time-delay of the single frames of media delivery will strengthen; And when the data interlacing degree that is acted on is less than normal, data interlacing will be not enough to deal with the data packet loss of burst.
But in the 3G network of low velocity, high packet loss, weak stability, usually there is unexpected error in the actual video communication system at present, causes losing in a large number of data.Through discover that these used interleaving modes are fixed singles at present in system, this is irrational for the video image of different quality, and the effect after interweaving is also unsatisfactory.
In view of above analysis, its problem that existing vedio data interweaves is summed up as follows:
1, is not suitable for interleaving treatment more greatly for the single frames data of video H.264, carries out certain flexible processing, could embody the effect that interweaves when interweaving, satisfy the fault-tolerant requirement of network it.
2, in the interweaving encoding process, select frequency coded sequence, single image, macro block, piece all to exist bigger problem as basic task scheduling unit.
3, in a system, be directed against the video of different quality and different transmission requirements, interleaver should be dynamic, can carry out rational algorithm according to the actual conditions of self and select.
Along with people's is to the further lifting of network quality, and many traditional Technology Needs are reformed and satisfied people to the high-quality requirement of communication in the network.
List of references
[1] Ye Yongtao, Zhang Shenru, imperial court is prosperous, and what ties up Design of Interleaver and performance thereof in the seedling Turbo code,
[2] Zhouning County million, Song Bin, Chang Yilin. based on the anti-packet loss method of video communication H.264/AVC research [J]. and Chinese journal of computers, 2006.29 (2): 267-273,
[3] Liu Zhiyong, Liu Xiaohong Tai Er laboratory catv terminal Department of Communication Force is standard and the application 2 006-07-10 in video meeting system thereof H.264,
[4] Song Bin, Guo Chunfang, Qin Hao is based on H.264 interweaving of video communication protected algorithm communication journal 2007-6-11.
Summary of the invention
Technical problem: the purpose of this invention is to provide a kind of method based on dynamic interleaving in the video transmission of slice level; The quality problems of video transmission have well been improved; Reduce the requirement of interleaver greatly, changed the single situation that can not carry out Dynamic Selection according to the video data volume of interleaver in the past room and time.
Technical scheme:, make data out of order through interweaving audio frequency, two groups of data of video through selecting interleaver that data are disperseed understanding on the basis of agreement H.264.Select the advantage of this method to be the concentrated mistake that burst in the transmission course produces is disperseed, make local a large amount of packet losses but satisfy the primary demand of media play under the situation about not retransmitting.But because H.264 the single frames data of video are bigger, if simply audio data frame and video data frame are not added directly interweaving of processing, it is also inevitable inadvisable to eliminate correlation.
This method is made up of video analyzer, section strategy, tactful three parts that interweave; The original video data data are the video analyzer that reads in of unit one frame, one frame with the frame, draw some parameters about video image clarity, image importance, image data amount through it; The section policy section be with section as basic task scheduling unit, be carrying out secondary based on slicing block and decompose at random on the basis of video analyzer, after confirming to be provided with after the parameter, piece image is divided into a limited number of section; Interweave policy section after receiving the data that transmit of section policy section, analyze the selection of dynamic algorithm of laggard row and interleave depth according to size, these parameter interleavers of number of section, and then realized data interlacing based on section.
The parameter that vedio data draws through video analyzer about image definition, image importance, image data amount; Wherein video analyzer is to be formed by motion estimation techniques, face recognition technology and these three kinds of technical combinations of image definition assessment; The result of video analyzer is that the weights of on the basis of the parameter of the parameter of the parameter that draws estimation respectively, recognition of face and image definition assessment, getting them obtain; The section policy section be meant food slicer adopt manager employee's mode dispatching algorithm be made into; Food slicer is confirmed the parameter setting after reading in source code flow, image is cut into a limited number of section; The policy section that interweaves is to confirm that by the selection of result, the algorithm that interweaves of section policy section and interleave depth three parts constitute.
The motion estimation techniques that relates in the video analyzer is to find the solution kinematic parameter and carry out part and overall situation adjustment parameters optimization value through the optimization aim function; In the layering iterative process, carry out preliminary classification; And fully take into account the attribute on the video image room and time; Realize moving object and cover accurately cutting apart of background, progressively accurately find the solution the global motion parameter according to classification results then; What face recognition technology adopted is a kind of recognition of face based on characteristic; Confirm the Primary Location of initial graphics face zone realization to be checked face earlier with simple operationss such as preliminary treatment; On the basis in this zone, utilize improved particle algorithm crowd's algorithm and Face Detection to obtain the different detection result, merge two results at last; The intelligibility evaluation of image is an employing acutance algorithm, utilizes some acutance algorithm to be easy to operation programization and evaluation and evaluation time are estimated all kinds of digital pictures rapidly and accurately than the characteristics of spatial domain degree parameters such as weak point with to the characteristics of the responsive frequency domain difference of the variation of gradation of image distributional difference definition; The interweave selection of algorithm of the policy section that interweaves comprises interweave two kinds of algorithm and the random interleaving algorithms of rule, and the concrete condition according to video image during application is selected; Wherein, the confirming of interleave depth is to be confirmed by the requirement of image transmitted quality, the decline under actual mobile communication environment, and with the sudden mistake that causes digital data transmission, interleave depth is big more, and then dispersion is big more, and anti-burst error ability is also just strong more.
Beneficial effect:
1) estimation, recognition of face, image definition are analyzed the prior art comparative maturity, and mutually combining between them is high to the precision of analysis of whole video image, and the data of this programme are dynamic changes, and these technology also satisfy time requirement.
2) being chosen at of slice also is dynamic here, and it reasonably selects the size and the number of cutting into slices according to the situation of image, saves space resources dynamically.
3) the efficient height that interweaves, the resource of saving room and time simultaneously.
4) be applicable to various based on the interleaving treatment in the video file transmission course H.264.
Description of drawings
Fig. 1, based on the block diagram of slice,
The illustraton of model of Fig. 2, dynamic interleaving device,
Fig. 3, design flow diagram.
Embodiment
The present invention proposes a kind of systems approach based on dynamic interleaving in the video transmission of slice level, has changed the situation that in the past interweaves directly so that frame data carry out and interleaver is single.This method relates to image definition assessment, recognition of face, three concrete technical parameters of estimation; On the basis of their three technical parameters, carry out weighting and get their weights, for the slice size is selected, the Dynamic Selection definite and interleaver of quantity provides important basis.
The method that the present invention is based on dynamic interleaving in other video transmission of slice (section) level is:
1) set up video analyzer: the parameter value of image definition assessment, the parameter value of recognition of face, the parameter value of estimation in the analytic record video image, its weights are got in these parameter value weightings later on, with this foundation as selection slice size.
2) Slice strategy: algorithm,, is encoded by each processor after video interweaving encoding task H.264 reasonably distributed and dispatch as basic task scheduling unit with Slice.According to the quality of video image with and the size of single frames data volume; Carrying out secondary based on the Slice piece and decompose at random on 1 basis analyzed; After confirming that parameter is provided with, piece image is divided into a limited number of Slice, and it is on average distributed to processor successively encode;
3) strategy that interweaves: after receiving 2 data; Size, number according to slice; Later carry out the selection of dynamic algorithm and interleave depth according to these parameter interleaver analyses, and then realized behind the slice that dynamic data interlacing has guaranteed to transmit the flexibility of multiple stream.
The implementation process of this programme can be summarized as follows:
1) video analyzer, all information of the statistics video image of broadcasting like situation such as Y-PSNR, image importance, image data amounts, are convenient to select the method for slice;
2) slice that receives video analyzer analysis result message partly starts the slice method, and in time regulates and control accordingly according to the result of video analyzer analysis;
3) interleaver partly receives data that slice partly transmits and according to the size of these data volumes and the quality requirement decision interweaving method and the interleave depth of image transmission.
Fig. 3 is this system work principle figure, and the work that each module is responsible for is following:
1, video analyzer design
1) estimation
Find the solution kinematic parameter and carry out part and overall situation adjustment parameters optimization value through the optimization aim function; We carry out preliminary classification in the layering iterative process; And fully take into account the attribute on the video image room and time; Realize moving object and covered accurately cutting apart of background, progressively accurately found the solution the global motion parameter according to classification results then.
2) recognition of face
A kind of simple recognition of face based on characteristic; At first utilizing straightforward procedure is that simple operations such as preliminary treatment confirms that initial circular face zone to be checked realizes the Primary Location of face; On the basis in this zone, utilize improved particle cluster algorithm and Face Detection to obtain the different detection result; Merge two results at last, so just guaranteed the correctness that detects significantly, improved detection speed simultaneously because of dwindling of surveyed area.
3) image definition assessment
Point acutance algorithm; The point acutance not only possesses operation programization and the characteristics of evaluation time than spatial domain grey parameters such as weak points of being easy to; Also possess simultaneously the characteristics that the gradation of image distributional difference changed responsive frequency domain parameter, thereby can estimate the definition of all kinds of digital pictures quickly and accurately.
2, slice strategy design
H.264 the single frames data of video are bigger, intend carrying out secondary for the video single frames after decomposing based on the Slice piece and decomposing, with Slice as basic task scheduling unit.Server reads in original video code flow in a certain order, according to the analysis result of video analyzer and the requirement of transmission of video images quality video image is being carried out the slice section, again through the pairing position of each Slice, pointer location.
3, interweave strategy design
1) algorithm that interweaves is selected: regular interleaver is to be applied to the earliest in the chnnel coding; It is exactly usually our said block interleaved device in fact; Just row is read to list or be listed as to read the interleaver that row goes out, and wherein regular data interleaver and irregular data interleaver realize that simply algorithm complex is lower; In the short communication system of transmission information frame, use Turbo code, adopt the block interleaved device more favourable.And that random interleaver produces the decentralization of data is higher, can solve the burst packet loss to greatest extent, the performance at random of random interleaver depend primarily on random number producing method, interleaver major parameter S, value the aspect such as choose.
2) confirming of interleave depth: the decline under actual mobile communication environment, with the sudden mistake that causes digital data transmission, interleave depth is big more; Then dispersion is big more; Anti-burst error ability is also just strong more, but interleave depth is big more, and the interweaving encoding processing time is long more; Thereby cause data transmission delay to increase, interweaving encoding is cost with time.
3) on the basis of slice strategy with it according to predetermined order, the transmission data command by among the messaging environment MPI is distributed to the interleaver node; Interleaver receives data; And encode, the data behind the coding also send to server via the transmission data command among the messaging environment MPI, after server is confirmed to collect the data of interleaver device node transmission; Assemble; Form video bit stream, read down the original video code stream of piece image simultaneously, cut into slices once more, interweave, work such as assembling.