CN101316198B - Monitoring method for multimedia data based on index - Google Patents
Monitoring method for multimedia data based on index Download PDFInfo
- Publication number
- CN101316198B CN101316198B CN 200710111645 CN200710111645A CN101316198B CN 101316198 B CN101316198 B CN 101316198B CN 200710111645 CN200710111645 CN 200710111645 CN 200710111645 A CN200710111645 A CN 200710111645A CN 101316198 B CN101316198 B CN 101316198B
- Authority
- CN
- China
- Prior art keywords
- data
- medium data
- index
- video
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 30
- 238000012544 monitoring process Methods 0.000 title claims abstract description 24
- 230000005540 biological transmission Effects 0.000 claims description 16
- 238000011156 evaluation Methods 0.000 claims description 11
- 238000007906 compression Methods 0.000 claims description 5
- 230000006835 compression Effects 0.000 claims description 5
- 238000005070 sampling Methods 0.000 claims description 3
- 230000002349 favourable effect Effects 0.000 abstract 1
- 238000005516 engineering process Methods 0.000 description 3
- 238000013441 quality evaluation Methods 0.000 description 2
- 241001269238 Data Species 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013144 data compression Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Images
Landscapes
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a method for monitoring multimedia data based on index, the steps are as follows: indices between a transmitted multimedia data identification number and corresponding multimedia characteristic data is established at a server terminal; the indices include data package numbers, data types or index data; the data package number means the multimedia data identification number that is received or transmitted; the data type means the type of the multimedia data package, the index data means the multimedia characteristic data corresponding to the data package number; after feeding back the multimedia data identification number corresponding to the false multimedia data package to the server terminal, a receiving terminal looks for the multimedia characteristic data corresponding to the multimedia data identification number in the indices of the server terminal and monitors multimedia service performance according to the multimedia characteristic data. By establishing the indices, the invention simplifies the reconstruction process of the multimedia data, thus being favorable for monitoring the multimedia service performance to more than one receiving terminals at the server terminal simultaneously.
Description
Technical field
The present invention relates to transmission of a kind of multimedia messages and treatment technology, specifically, relate to a kind of monitoring method of the multi-medium data based on index.
Background technology
Along with the development of cable network and wireless network and the driving of user's request, multimedia technology has obtained increasing concern with relevant application, is considered to one of mainstream applications of following express network.Multimedia application comprises Video Applications and voice applications, and is with respect to conventional use such as WWW, the E-mail etc. of Internet, bigger to the demand of bandwidth.The coding techniques of high compression rate has been taked in the transmission of multi-medium datas such as existing video and audio frequency, and these coding techniquess have utilized multimedia messagess such as video and audio frequency to carry out data compression in time, space and statistical dependence.Multi-medium data after the compression is very responsive to error of transmission; Particularly existing wired or wireless network does not provide the QoS (Quality ofServerices) of transmission to guarantee; The losing or postpone unavoidably of packet, this all can cause the user experience quality of receiving terminal to descend.Therefore multimedia service performance is monitored and become the research of field of multimedia communication focus.
In traditional monitoring multimedia service performance method, server end is packed to multi-medium data, and sends it to receiving terminal, during to pass through the plurality of network node.In the transport process of multi-medium data, the multi-medium data bag may take place to postpone or lose, and causes receiving terminal can not correctly receive all packets.At this moment, receiving terminal is handled through multimedia decoding, finds the error message in the content of multimedia, and it is fed back to server end.Monitor server according to the feedback error information that receives, utilizes multimedia decoder to estimate monitored receiving terminal multi-medium data again.With monitored multi-medium data with compare with reference to multi-medium data, realize the monitoring of multimedia service performance.
The shortcoming that said method exists comprises:
One, need carry out multimedia decoding at receiving terminal and handle, the error message in the content of multimedia that could cause owing to the Network Transmission mistake feeds back to server end, is unfavorable for the protection of digital publishing rights.
Two, when server end carries out the multimedia service performance evaluation, need carry out complete multimedia decoding to each monitored receiving terminal and handle.Therefore when a server end is monitored a plurality of receiving terminals simultaneously, can take a large amount of computational resources.
Summary of the invention
The technical problem that the present invention solved provides a kind of monitoring method of the multi-medium data based on index, can realize the monitoring and the evaluation of multimedia service performance easily.
Technical scheme is following:
A kind of monitoring method of the multi-medium data based on index, step is following:
(1) index between multi-medium data identification number that server end set up to send and corresponding multimedia characteristic; Comprise packet numbers, data type or index data in the said index; Said packet numbers is represented the multi-medium data identification number that receives or send, said data types to express multi-medium data bag type, and said index data is represented the multimedia characteristic corresponding with packet numbers;
(2) after the multi-medium data identification number that receiving terminal is corresponding with wrong multi-medium data bag feeds back to server end; In the said index of server end, search the multimedia characteristic corresponding, carry out multimedia service performance according to said multimedia characteristic and monitor with said multi-medium data identification number.
Further, in the step (1), the multi-medium data identification number of said transmission is the sequence number that coding back multi-medium data sends, and in Real-time Transport Protocol, the multi-medium data identification number of transmission is a RTP bag sequence number; Perhaps, in ICP/IP protocol, the multi-medium data identification number of transmission is an IP bag sequence number; Perhaps, in agreement H.324M, the multi-medium data identification number of transmission is the Frame sequence number.
Further, in the step (1), said multimedia characteristic comprises video data or the voice data after the compression.
Further, in the step (1), said index data comprises video data or voice data; Corresponding with it, said data type is video or audio frequency.
Further; In the step (1); Said video data comprises frame number, macro block information or the residual information of video parameter, video frame image, and said video parameter is meant the video sequence coding parameter information, and the frame number of said video frame image is meant the sequence number of the video frame image that current multi-medium data bag is corresponding; Said macro block information is meant the image block characteristic that is used for the reconstruction video picture frame, and said residual information is meant the predicted value of the inner picture element of image block and the differential data between the decode value; Said voice data comprises the frame number of audio frequency parameter or audio data frame, and said audio frequency parameter is meant the coding parameter of voice data, and the frame number of said audio data frame is meant the sequence number of current multi-medium data bag corresponding audio Frame.
Further, in the step (1), said video parameter comprises figure image height, figure image width, video frame rate or video format; Said macro block information comprises the type of image block, the numbering of image block, the movable information or the reference frame information of image block; Said audio frequency parameter comprises the sample rate and the sampling interval of speech coding.
Further, in the step (2), said multi-medium data identification number is the sequence number of packet.
Further, step (2) comprising:
(21) according to the said index of the sequence number search of misdata bag;
(22) in said index, search the packet numbers corresponding with the sequence number of said misdata bag, and the multimedia characteristic corresponding with said packet numbers;
(23) carry out the evaluation and the monitoring of multi-medium data based on said multimedia characteristic.
The inventive method will be rebuild the back multi-medium data and compared with reference to multi-medium data, obtain the evaluation index of multimedia service performance, can realize the monitoring and the evaluation of multimedia service performance easily through evaluation index.This method is through the index between the Data Identification that set up to send number and the multi-medium data; Thereby the process of reconstruction of the multi-medium data when having simplified the multimedia service quality evaluation helps simultaneously a plurality of receiving terminals being carried out monitoring multimedia service performance at server end.
Description of drawings
Fig. 1 is the structural representation of index;
Fig. 2 is that the piece in is H.264 divided sketch map;
Fig. 3 is the process of reconstruction sketch map of mistake video sequence.
Embodiment
The present invention is through the index between the Data Identification that set up to send number and the multi-medium data, thereby the process of reconstruction of the multi-medium data when having simplified the multimedia service quality evaluation helps at server end a plurality of receiving terminals being carried out monitoring multimedia service performance.
With reference to the accompanying drawings, the preferred embodiments of the present invention are described in detail.
Index between the first step, the multi-medium data identification number that set up to send and corresponding multimedia messages.
The multi-medium data identification number that sends is the sequence number that coding back multi-medium data sends, and in Real-time Transport Protocol, the multi-medium data identification number of transmission is a RTP bag sequence number; In ICP/IP protocol, the multi-medium data identification number of transmission is an IP bag sequence number; In agreement H.324M, the multi-medium data identification number of transmission is the Frame sequence number.
Multimedia messages can be video or the voice data after the compression, perhaps other data type.
As shown in Figure 1, the index of foundation should comprise (being not limited to) following information:
1, packet numbers (Packet No.): the multi-medium data identification number that expression receives or sends, this packet numbers can be used as the search key of index, retrieves the multi-medium data that needs.
2, data type (Packet Type): represent the multi-medium data bag type of current index, this data type can be video or audio frequency (Video Type/Audio Type).
3, index data (Index Data): represent the multi-medium data corresponding with packet numbers.For different multimedia data types (video or audio frequency), the content of index data is also different.
A, for the index data of video type, index data should comprise frame number, macro block information and the residual information etc. of video parameter, video frame image.Wherein macro block information and residual information can be used as optional index data content.
(1) video parameter (Video Parameters) is meant the video sequence coding parameter information, for example schemes contents such as image height, figure image width, video frame rate, video format.
(2) frame number of video frame image (Video Frame No.) is meant the sequence number of the video frame image that current multi-medium data bag is corresponding.
When including a plurality of picture frame in the video packets of data, index data should include the frame number of a plurality of picture frames.
(3) macro block information (Macro_block Information) is meant the image block characteristic that is used for the reconstruction video picture frame, the for example movable information of the numbering of the type of image block, image block, image block, reference frame information etc.
When including a plurality of macro block in the video packets of data, macro block information should include the characteristic of a plurality of macro blocks.One two field picture consists of a plurality of macro blocks.
(4) residual information (Residual Information) is meant the characteristic that is used for the reconstruction video picture frame, comprises the predicted value of each picture element that image block is inner and the differential data between the decode value.
As shown in Figure 2, H.264 to be example, a two field picture (Picture) comprises a plurality of bands (Slice), and each band comprises a plurality of macro blocks (Macroblock).Wherein each macroblock size is 16 * 16 pixels, and it is divided into the sub-piece (Block) of 16 4 * 4 pixels again.When the storage macro block information, need write down the position of each macro block in image, and the motion vector of every sub-block, sub-block type and reference frame information.
H.264 to be example, when a macro block includes 16 4 * 4 sub-pieces of pixel, need the residual information of 16 pixels in each 4 * 4 sub-piece of record.
B, for audio types, index data should comprise the frame number of audio frequency parameter and audio data frame.
(1) audio frequency parameter (Audio Parameters) is meant the coding parameter of voice data.The for example sample rate of speech coding and sampling interval etc.
(2) frame number of audio data frame (Audio Frame No.) is meant the sequence number of current multi-medium data bag corresponding audio Frame.When including a plurality of audio frame in the packets of audio data, index data should include the frame number of a plurality of audio frames.
Second step is based on the reconstruction of the multi-medium data of index data.
In index, set up the corresponding relation between index data and the multi-medium data identification number, therefore can rebuild multi-medium data according to index data.When receiving terminal with the data mode information that receives; This data mode information comprises the multi-medium data identification number (being the sequence number of misdata bag in the present embodiment) of misdata bag; After the sequence number of misdata bag fed back to server end; Just can retrieve corresponding multimedia messages, thereby carry out the reconstruction of multi-medium data according to index data.With the multi-medium data of rebuilding with compare with reference to multi-medium data, obtain the multimedia service performance evaluation index, be the multi-medium data of correctly decoding with reference to multi-medium data.
Concrete multi-medium data method for reconstructing is following:
1, server end obtains the data mode information of multi-medium data bag.
If packet drop appears in receiving terminal, this data mode information comprise lose the sequence number of packet.
The sequence number of 2, losing packet according to this institute is then searched the packet numbers in the index, retrieves corresponding index data according to the indexed data Bale No..The state information of multi-medium data bag can be obtained through feedback channel.
3,, rebuild the multi-medium data of makeing mistakes according to index data.
If current data is a video type, then carries out video image and rebuild.
(1) confirms non-affected area and affected area according to the video index data.
As shown in Figure 3, suppose two frame of video (comprise previous I frame, do not comprise a back I frame) between the I frame as a GOP (Group Of Picture), I1, P2, P3, B4, P5 ... before next I frame was a GOP.In current GOP, finding the frame (like the P3 among Fig. 3) at first packet loss place among this GOP according to index data and packet loss information, do not have mistake (like I1 Fig. 3 and P2) from the I frame of this GOP to all frames before this packet loss place frame, is non-affected area.In packet loss place frame (P3), affected area is the black part, and corresponding to the position at lost data packets place, the blank parts among the P3 is non-affected area.In the subsequent frame that receives this erroneous effects, like B4, P5, finish up to current GOP, according to the wherein motion vector and the reference frame information of relevant block, judge affected area, shown in the part of the striped among B4, the P5.Other blank parts is non-affected area.
(2) video image of non-affected area is rebuild, directly the correspondence position from reference video sequence duplicates and obtains.
(3) video image of affected area being rebuild, can be 0 to the affected area indirect assignment; Also can the correspondence position from reference video sequence duplicate and obtain; Also can obtain according to motion compensation process according to image feature data (macro block information and residual information); Perhaps obtain according to other video error coverage method.
If current data is an audio types, then carries out voice data and rebuild according to the audio frequency method for reconstructing.
(1) confirms non-damaged data and damaged data according to the audio index data.
(2) non-damaged data is rebuild, can directly from the reference audio data, be duplicated and obtain.
(3) damaged data being rebuild, can be 0 to the affected area indirect assignment; Perhaps obtain according to other lost packet compensation method.
The 3rd step, with the multi-medium data after rebuilding with compare with reference to multi-medium data, obtain the evaluation index of multimedia service performance.
Behind the multi-medium data that obtains rebuilding, can and compare it with reference to multi-medium data, obtain the evaluation index of multimedia service performance.For example, when the Video service performance was estimated, the Y-PSNR (Peak Signal to Noise Ratio) that can use reconstructed image and reference picture was as evaluation index.
Claims (8)
1. monitoring method based on the multi-medium data of index, step is following:
(1) index between multi-medium data identification number that server end set up to send and corresponding multimedia messages; Comprise packet numbers, data type and index data in the said index; Said packet numbers is represented the multi-medium data identification number that receives or send, said data types to express multi-medium data bag type, and said index data is represented the multimedia messages corresponding with packet numbers;
(2) after the multi-medium data identification number that receiving terminal is corresponding with wrong multi-medium data bag feeds back to server end; Server end is searched in said index and the corresponding multimedia messages of said multi-medium data identification number; Rebuild the multi-medium data of makeing mistakes according to said multimedia messages, and then carry out monitoring multimedia service performance.
2. the monitoring method of the multi-medium data based on index according to claim 1; It is characterized in that in the step (1), the multi-medium data identification number of said transmission is the sequence number that coding back multi-medium data sends; In Real-time Transport Protocol, the multi-medium data identification number of transmission is a RTP bag sequence number; Perhaps, in ICP/IP protocol, the multi-medium data identification number of transmission is an IP bag sequence number; Perhaps, in agreement H.324M, the multi-medium data identification number of transmission is the Frame sequence number.
3. the monitoring method of the multi-medium data based on index according to claim 1 is characterized in that in the step (1), said multimedia messages comprises video data or the voice data after the compression.
4. the monitoring method of the multi-medium data based on index according to claim 1 is characterized in that in the step (1), said index data comprises video data or voice data; Corresponding with it, said data type is video or audio frequency.
5. the monitoring method of the multi-medium data based on index according to claim 4; It is characterized in that; In the step (1); Said video data comprises frame number, macro block information or the residual information of video parameter, video frame image, and said video parameter is meant the video sequence coding parameter information, and the frame number of said video frame image is meant the sequence number of the video frame image that current multi-medium data bag is corresponding; Said macro block information is meant the image block characteristic that is used for the reconstruction video picture frame, and said residual information is meant the predicted value of the inner picture element of image block and the differential data between the decode value; Said voice data comprises the frame number of audio frequency parameter or audio data frame, and said audio frequency parameter is meant the coding parameter of voice data, and the frame number of said audio data frame is meant the sequence number of current multi-medium data bag corresponding audio Frame.
6. the monitoring method of the multi-medium data based on index according to claim 5 is characterized in that, in the step (1), said video parameter comprises figure image height, figure image width, video frame rate or video format; Said macro block information comprises the type of image block, the numbering of image block, the movable information or the reference frame information of image block; Said audio frequency parameter comprises the sample rate and the sampling interval of speech coding.
7. the monitoring method of the multi-medium data based on index according to claim 1 is characterized in that in the step (2), said multi-medium data identification number is the sequence number of packet.
8. the monitoring method of the multi-medium data based on index according to claim 7 is characterized in that step (2) comprising:
(21) according to the said index of the sequence number search of misdata bag;
(22) in said index, search the packet numbers corresponding with the sequence number of said misdata bag, and the multimedia messages corresponding with said packet numbers;
(23) rebuild the multi-medium data of makeing mistakes based on said multimedia messages, and then carry out the evaluation and the monitoring of multi-medium data.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 200710111645 CN101316198B (en) | 2007-06-01 | 2007-06-01 | Monitoring method for multimedia data based on index |
| PCT/CN2008/000421 WO2008131636A1 (en) | 2007-04-29 | 2008-03-03 | A method for measuring the multimedia data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 200710111645 CN101316198B (en) | 2007-06-01 | 2007-06-01 | Monitoring method for multimedia data based on index |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101316198A CN101316198A (en) | 2008-12-03 |
| CN101316198B true CN101316198B (en) | 2012-07-04 |
Family
ID=40107055
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 200710111645 Active CN101316198B (en) | 2007-04-29 | 2007-06-01 | Monitoring method for multimedia data based on index |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101316198B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110088076A1 (en) * | 2009-10-08 | 2011-04-14 | Futurewei Technologies, Inc. | System and Method for Media Adaptation |
| CN109147914B (en) * | 2018-10-16 | 2023-04-28 | 上海联影医疗科技股份有限公司 | Image reconstruction system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2005192073A (en) * | 2003-12-26 | 2005-07-14 | Matsushita Electric Ind Co Ltd | Multimedia data communication method and apparatus |
| CN1669324A (en) * | 2002-07-15 | 2005-09-14 | 汤姆森特许公司 | Adaptive Weighting of Reference Picture in Video Decoding |
| CN1692653A (en) * | 2002-11-22 | 2005-11-02 | 株式会社东芝 | Moving picture encoding/decoding method and device |
| CN1873820A (en) * | 2006-05-18 | 2006-12-06 | 北京中星微电子有限公司 | Method for processing video file, method for indexing video, and video indexing unit |
-
2007
- 2007-06-01 CN CN 200710111645 patent/CN101316198B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1669324A (en) * | 2002-07-15 | 2005-09-14 | 汤姆森特许公司 | Adaptive Weighting of Reference Picture in Video Decoding |
| CN1692653A (en) * | 2002-11-22 | 2005-11-02 | 株式会社东芝 | Moving picture encoding/decoding method and device |
| JP2005192073A (en) * | 2003-12-26 | 2005-07-14 | Matsushita Electric Ind Co Ltd | Multimedia data communication method and apparatus |
| CN1873820A (en) * | 2006-05-18 | 2006-12-06 | 北京中星微电子有限公司 | Method for processing video file, method for indexing video, and video indexing unit |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101316198A (en) | 2008-12-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101174995B (en) | Method and system for monitoring multimedia service performance | |
| CN102143367B (en) | Method, device and system for checking error correction | |
| KR101722719B1 (en) | Backward looking robust header compression receiver | |
| CN106937121B (en) | Image decoding and encoding method, decoding and encoding device, decoder and encoder | |
| CN110519635B (en) | Audio and video media stream converging method and system of wireless cluster system | |
| US8953671B2 (en) | Codec capability negotiation method and terminal thereof | |
| WO2020006912A1 (en) | Method and device for analyzing network transmission quality, computer equipment and storage medium | |
| KR102170717B1 (en) | Method and apparatus of rate adaptation utilizing ber for multimedia service | |
| US20150110168A1 (en) | Video data transmission method and apparatus | |
| CN106549916A (en) | Multicast transmission method, apparatus and system | |
| CN104270594B (en) | The method and apparatus that data packet sends and receives | |
| CN105554513A (en) | Panoramic video transmission method and system based on H.264 | |
| CN107534798A (en) | Broadcast singal sends equipment, broadcasting signal receiving, broadcast singal sending method and broadcast signal received method | |
| RU2673102C2 (en) | Device and method of information processing and program | |
| CN105979284B (en) | Mobile terminal video sharing method | |
| CN101316198B (en) | Monitoring method for multimedia data based on index | |
| CN101296166B (en) | Index-based multimedia data measurement method | |
| CN101179353A (en) | Method and system of monitoring multimedia service performance | |
| CN117176952A (en) | Video encapsulation and decapsulation method, apparatus, electronic device, and computer-readable storage medium | |
| CN104333765A (en) | Processing method and device of video live streams | |
| US20150271530A1 (en) | Video quality enhancement method and apparatus | |
| CN115834926B (en) | Video encryption method based on H.265 entropy coding binarization | |
| CN115842919B (en) | Video low-delay transmission method based on hardware acceleration | |
| CN109783475B (en) | Method for constructing large-scale database of video distortion effect markers | |
| CN1316829C (en) | Visual frequency flow coding/decoding method based on ripple convertion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |