KR20010106411A - Universal quality measurement system for multimedia and other signals - Google Patents

Abstract

In a method for measuring the perceptual quality of audio signals, auditory signals, auditory-visual signals, and multimedia signals in communication equipment, the system 10 receives a test signal processed via a network from the first device 26. Communicate with the second device 16. The processed test signal is then received by the device under test, which further processes the test signal to restore the display of the test signal. The recovered test signal is then objectively analyzed to determine the measure of perceptual quality by comparing the recovered test signal with a display of previously stored test signals. The quality measuring unit 12 properly attached to the equipment under test identifies and evaluates the quality of the recovered test signal communicated from the remote device or the test signal communicated by the equipment under test. In the latter case, since the quality measuring unit communicates with both the equipment under test and the network, the quality measuring unit itself receives an indication of the test signal communicated by the equipment under test.

Description

UNUALRSAL QUALITY MEASUREMENT SYSTEM FOR MULTIMEDIA AND OTHER SIGNALS

A number of processing and transmission methods have been devised for the communication of voice, auditory, visual and multimedia information. This type of signal is referred to herein as a "Voice, Audio, Video (VAV)" signal. The VAV signal may be processed in various ways before being sent to the receiver. For example, the VAV signal can be digitized, compressed, modulated and carried. The receiver then changes the received signal to return to the indication of the perceivable VAV signal.

Typically, the quality of the received VAV signal is evaluated by subjective tests. However, this type of test is not practical as a test method in service and does not produce consistent and reproducible results. Signal quality assessment is further complicated when VAV communications are integrated with very complex transmission systems such as cellular code division multiple access (CDMA) systems or wideband CDMA systems. When evaluating VAV quality in a nondeterministic environment such as the Internet, different but similar difficulties arise.

A method of indirectly evaluating signal quality by measuring parameters such as signal-to-noise ratio (S / N), carrier-to-interference ratio (C / I), lost packet ratio, and bit error rate (BER) is known. However, associating these parameters with the user's perception of the quality of the received VAV signal becomes difficult, especially when such signals are highly compressed or processed. In addition, these parameters are not suitable for accurately and accurately reflecting VAV quality over time.

Algorithms are known for appreciating sound quality. For example, the International Telecommunications Union (ITU) standard P.861 is an objective algorithm that can be used to automatically calculate qualitative numbers, such as Mean Opinon Score (MOS) for sound transmission. However, this type of test requires both an initial signal and a test signal that are statically analyzed by the item of test equipment. Thus, for quality assessment to be done in the field with the actual signal, the test rig must integrate the network architecture and protocol into its hardware, software or firmware. Often, network architectures are not available or difficult to implement and expensive. Proprietary restrictions can even hinder its implementation.

In view of the above, a method and apparatus for evaluating the quality of a received VAV signal without requiring specific knowledge of the operation and configuration of the network, such as the transmission technique used and the network connection technique, is preferred. It is also desirable for the apparatus and method to provide automated quality measurements in real time or non real time, such that interactive field tests can be performed, for example. In addition, a common test platform for various types of VAV signals that are easily connected to the equipment under test is desirable. Moreover, it is desirable to provide a method and apparatus for providing a qualitative quality of reliable, repeatable and easily understandable service measurements for one-way or multi-hop or round trip measurements.

The present invention relates to a system for measuring the quality of a signal, and more particularly to a system for measuring the quality of a multimedia signal communicated via a network.

1 is a block diagram showing an embodiment of a quality measurement system for a VAV signal suitable for one-way quality measurement according to the present invention;

2 is a flow chart showing an embodiment of a quality measurement method applicable to the system of FIG.

3 is a block diagram showing an embodiment of a quality measurement system suitable for round trip quality measurement according to the present invention;

4 is a block diagram showing an embodiment of a quality measurement system suitable for two-way quality measurement according to the present invention.

In one embodiment, the present invention is a method for measuring the perceived quality of speech signals, auditory signals, auditory-visual signals, and multimedia signals in communication equipment. The processed test signal is communicated from the first device to the second device via a network. The processed test signal is then received by the device under test, which further processes the processed test signal to restore the display of the test signal. The recovered test signal is then compared with an indication of a previously stored test signal to objectively analyze and determine a measure of perceptible quality.

In another embodiment, the present invention is a quality measuring unit attached to equipment under test to identify and evaluate the quality of a recovered test signal. The test signal is communicated from the remote device and by the equipment under test. In the latter case, the quality measuring unit is configured to communicate with the equipment under test and the network to receive an indication of the test signal communicated by the equipment under test.

In the method and apparatus of the present invention, the quality of the received VAV signal is evaluated without specific knowledge of the network in communication with the equipment under test. Moreover, both real-time and non-real-time quality analysis is possible using a common test platform for various types of VAV signals. The present invention can be applied to various types of qualitative quality of service measurements, including one-way and multi-hop and round trip measurements.

In one embodiment of the present invention with reference to FIG. 1, a quality measurement system 10 for a VAV signal is shown with equipment 16 and network 22 under test. The quality measuring system 10 includes a local quality measuring unit 12 (LQMU). Port 14 of LQMU 12 is coupled to the equipment 16 (EUT) under test. In one embodiment, the EUT 16 is a mobile or cellular telephone, and the port 14 is connected to the EUT 16 using, for example, a set of "hand-free" or "car kit" electrical terminals (not shown). Electrically coupled. However, the present invention is not limited to the adaptation of mobile and cellular telephones.

The EUT 16 transmits and receives information with the network 22 via each of the communication paths 18 and 20. For example, in the case of a cellular telephone, the network 22 has a wireless network 24 and the communication paths 18 and 20 are wireless links. In addition, the remote reference data storage unit 26 (RRDSU) is coupled with the network 22 to operate. In one embodiment, the RRDSU 26 is a device for recording and playing reference signals, for example, coupled to the wireless network 24 via a wired network 28, which is a telephone answering device. Since a network interface adapter 30 (NIA) is provided, the signal output from the RRDSU 26 is adjusted and provided to the network 28 in a form suitable for transmission. Similarly, the signal transmitted from network 28 to RRDSU 26 is converted by NIA 30 into a form suitable for processing by RRDSU 26.

In one embodiment, a local storage device in the form of a local reference data storage unit 32 (LRDSU) is provided for the LQMU 12, which is coupled to the LQMU 12 via a second port 34. In another embodiment, a local memory, such as an electronic or magnetic memory, is embedded in the LQMU 12. The LQMU 12 provides an indication of the quality measurement 36 of the test signal received from the RRDSU 26 via the network 22 by the EUT 16.

The knowledge of the protocols, signaling and configuration of the network 22 required for the practice of the present invention will be isolated within the EUT 16 and part of existing equipment and within the NIA 30. In most cases, NIA 30 may be an readily available device supplied by an operator or electronic distributor of network 28. The present invention allows various types of networks 22 to be used without the need for specific information about the nature of the network 22 that is known or available for RRDSU 26 or LQMU 12 or local storage 32. Allow. This allows the RRDSU 26 to be coupled via the NIA 30 to a public or user accessible service access point of the network 22.

In one embodiment, port 14 and LQMU 12 are configured to gather test signals received from EUT 16 such that LQMU 12 responds to signals received from EUT 16. In one embodiment, port 14 and LQMU 12 are configured to control EUT 16 to transmit an uplink signal via radio link 18. In addition, the LQMU 12 is configured to analyze the received test signals and to perform objective quality measurements of these signals. In one embodiment, algorithms may be configured and selected in the LQMU 12 for one or more VAV signal types.

The RRDSU 26 is, for example, a storage unit including a recorder and a player (not shown). The recorder may be, for example, a mass storage device such as a hard disk drive capable of reading / writing, or other device suitable for recording VAV signals. In one embodiment, the player connects the recorded VAV signal including the reference VAV signal and reproduces this signal. In one embodiment, the RRDSU 26 includes a controller module (not shown) that can be controlled by signals received via the network 22, wherein the RRDSU 26 starts, stops, Commands to perform a play, record, stop, search, timestamp, and so on. In one embodiment, the RRDSU 26 is equipped with a digital response device to perform these functions. In one embodiment, the RRDSU 26 is configured to transmit a test signal selected from voice, auditory, visual and multimedia test signals.

Referring to FIG. 2, the quality measurement system 10 tests the EUT 16 associated with a network 22 having a radiotelephone network 24 to test one-way downlink end-to-end VAV quality, such as network 24. Evaluate the quality of the signal received by the EUT 16 via the radio link 20 from the base station (not shown). First, since the LQMU 12 is turned on (100) and connected to the EUT 16, the LQMU responds upon receipt of the test message by the EUT 16. EUT 16 dials the RRDSU telephone number to call RRDSU 26 (102), causing RRDSU 26 to send a test signal to EUT 16. In another embodiment, when the EUT 16 is configured to communicate via a network other than the telephone network, telephone calling the RRDSU 26 is replaced with a suitable signaling technique. In another embodiment, such signaling may be performed manually, such as by dialing a number on the keypad (not shown) of the EUT 16.

In one embodiment, RRDSU 26 is configured as a voice mailbox, so dialing RRDSU 102 dials the voice mailbox of RRDSU 26 and retrieves 104 a test signal as a voice mail message. It will include. Prior to the transmission of a test signal, such as a voice message, the test signal is in a form suitable for processing by the NIA 30 to be transmitted across the network 28. The LQMU 12 then attempts to synchronize with the test signal (106). Sync search 106 continues until sync 108 is achieved. In one embodiment, an error condition 112 is signaled when synchronization 108 is not achieved within the determined time period 110. In one embodiment, by synchronizing with the test signal 108, the LQMU 12 analyzes the signal received by the EUT 16 to determine whether the test signal has been received. The EUT 16 further processes the signal received via the network 24 to restore the display of the test signal analyzed by the EUT 16. Further processing the received signal may allow the EUT 16 to be independent of the signal encoding and network protocol used by the network 22.

After acquiring synchronization (108), the recovered test signal is received by the LQMU 12 (114). The received test signal is compared with the indication of the reference signal previously stored in the LRDSU 32 and the measurement of the quality of service (QoS) is determined so that an objective measurement of the received signal quality is provided (116). This measurement or indication of the measurement is displayed, for example, on a display screen (not shown) (118). In another embodiment, the quality measurement is recorded for future display or analysis. In one embodiment, a set number of tests are performed during a call to RRDSU 26 (120). The call from the EUT 16 to the RRDSU 26 may be terminated manually or automatically (122) when the determined test or multiple tests are completed.

In one embodiment, if the LQMU 12 does not have sufficient processing power to evaluate the quality of the test signal received by the EUT 16, the received test signal is later, such as from the RUTSU 26 from the EUT 16. Is stored in the LRDSU 32 for offline quality assessment after the call to.

In another embodiment of the present invention with reference to FIG. 3, the uplink test system 200 does not require an RRDSU 26. Test system 200 is shown in FIG. 3 with network 22 and EUT 16. In this embodiment, the EUT 16 is a cellular telephone that transmits to the wireless network 24 via the radio link 18. The LQMU 12 receives a signal from the NIA 30 via a wired network 26. In one embodiment, the NIA 30 is a modem, such as a computer modem, configured to communicate with the LQMU 12. EUT 16 dials the telephone number of NIA 30 to initiate the test. The LQMU 12 then injects a local-to-remote test signal into the EUT 16, for example via a hand-free kit, which is routed to the LQMU 12 via the network 22 and the NIA 30. Is returned. The LQMU 12 then performs a quality of service assessment based on the received signal.

Another embodiment of the present invention with reference to FIG. 4 is provided with a two-way test system 300 having a master quality measuring unit 302 (MQMU) and a slave quality measuring unit 304 (SQMU). The two-way test system 300 represents a configuration in which a test of the EUTs 306, 308 is performed with the network 310. The MQMU 302 is configured to initiate and harmonize the communication between the EUT 306 and the EUT 308, for example a terminal, that provides VAV capable output via a network 310 that is, for example, the Internet, and the MQMU 302. And timestamps for test results generated by both SQMU 304. In addition, similar to the LQMU 12, the MQMU 302 and the SQMU 304 are configured, and the MQMU 302 is coupled to the EUT 306 and the reference data storage unit 312 (RDSU), and the SQMU 304 Is coupled to the EUT 308 and the RDSU 314. Both MQMU 302 and SQMU 304 are configured to provide a quality measure of the received signal. The MQMU 302 provides a quality measure 316 of the signal transmitted from the EUT 308 to the EUT 306 via the network 310, while the SQMU 304 provides the EUT 308 from the EUT 306. A quality measurement 318 of the signal transmitted via the raw network 310 is provided. In one embodiment, the time stamping of the quality measurement is determined so that the quality measurements 316 and 318 can be correlated and later processed by the post processing module 320.

In one embodiment, the LQMU 12, the RRDSU 26, the MQMU 302, or the SQMU 304 do not require a direct connection with a given network. For this reason, each of these devices can operate independently of the network type, including its characteristics and configuration, thus allowing use with both proprietary and non-exclusive networks. As a result, the present invention may be practiced with various types of networks, including switch circuits, packet switches, frame relays, the Internet, protocols, and asynchronous transfer mode (ATM) networks. In addition, the present invention is suitable for various network transmission technologies including wireless, wired and satellite networks. Moreover, the components of the present invention can be designed to connect to existing terminals or ports of communication equipment and to service access points of public or user available networks. Thus, no specific setup or modification of the network or equipment under test is required. In various embodiments, some or all of the LQMU 12 and the MQMU 302 and the SQMU 304 are configured to not affect each other so that they are deployed for service.

In one embodiment, real time quality evaluation is achieved since at least one of the LQMU 12, the MQMU 302, and the SQMU 304 is implemented using digital signal processing. If network architecture or protocol information is available, the quality measurements obtained are used to adapt the network parameters to adaptability.

In another embodiment, speech quality judgment is improved by, for example, adapting or automatically selecting a suitable algorithm for quality judgment. For example, multi-media signals transmitted via CDMA and Voice Over IP can be used to measure voice quality offline using a file stored on a computer. It is better evaluated by algorithms different from standard quality measurement algorithms such as measurement (PSQM), standardized block measurement (MNB) or other ITU developed measurement algorithms. In one embodiment, at least one of LQMU 12 and MQMU 302 and SQMU 304 are programmable for various system applications. In one embodiment, various test signals are selected manually or under control of at least one of the LQMU 12 and the MQMU 302 and the SQMU 304. Designed by a standard generating body such as an ITU, for example, an artificial pseudo-voice signal can be selected.

In one embodiment, the storage devices 32, 310, 312 include relay devices that communicate with various types of storage media or other devices having the storage media.

In one embodiment, at least one of the LQMU 12, the MQMU 302, and the SQMU 304 may operate on at least one of a Global Positioning System (GPS) receiver (not shown) and a digital map (not shown). Can be combined to automatically generate a quality of service map such that test signals, for example for a radiotelephone network, are repeatedly transmitted to the moving EUT 16,306 or 308, and performance measurements are performed and analyzed as a function of location.

It is known to those skilled in the art that other modifications can be made without departing from the spirit of the invention. Therefore, the scope of the present invention is determined with reference to the following claims.

Claims (30)

A method for measuring the perceptual quality of at least one of an audio signal, an audio signal, an auditory-visual signal, and a multimedia signal of a communication device in an operating environment, the method comprising: Transmitting a voice signal, a processed test signal which is at least one of an auditory signal, an auditory-visual signal, and a multi-media signal from a first device via a communication network; Receiving the processed test signal using the second device, Processing the processed test signal to restore the display of the test signal, And analyzing the perceptual quality of the recovered test signal by comparing the recovered test signal with a representation of a previously stored test signal. 2. The apparatus of claim 1, wherein the first apparatus has a remote data storage unit (RRDSU) operatively coupled to the network for communicating the processed test signal, and the second apparatus comprises a network for receiving the processed test signal. And an item of equipment under test (EUT) operatively coupled to a quality measuring unit (QMU) for analyzing the quality of the recovered test signal, Prior to the step of analyzing the quality of the recovered test signal, further comprising analyzing the recovered signal by the EUT to determine that the test signal is to be recovered. 3. The method of claim 2, wherein the QMU further comprises displaying an indication of the quality of the recovered test signal. 3. The method of claim 2, further comprising contacting the EUT and RRDSU via a network prior to the step of communicating the processed test signal. 5. The method of claim 4, wherein the network comprises a telephone network, the EUT comprises a telephone, and wherein contacting the EUT and the RRDSU via a network further comprises dialing a telephone number of an RRDSU. Method for measuring perceptual quality. 6. The method of claim 5, wherein contacting the EUT and RRDSU further comprises dialing a voice mailbox access code. 3. The method of claim 2, wherein analyzing the quality of the recovered test signal is performed while a processed test signal is received by an EUT. The method of claim 2, further comprising storing the recovered test signal, wherein analyzing the quality of the recovered test signal is performed after the test signal is stored. . 9. The method of claim 8, further comprising disconnecting an EUT from a network prior to analyzing the quality of the recovered test signal. 3. The method of claim 2, wherein analyzing the quality of the recovered test signal is performed digitally. 3. The method of claim 2, further comprising selecting a quality determination method according to the characteristics of the network. The method of claim 2, wherein the network is a radiotelephone network, and the steps of communicating the processed test signal and analyzing the quality of the recovered test signal are repeatedly performed. Moving the EUT such that repeated communication of the processed test signal is received at the EUT at various locations; And analyzing the quality of the recovered test signal as a function of the EUT at various locations. 3. The processed test signal of claim 2, wherein the processed test signal communicated by the first device to the second device is a processed indication of the first test signal, Injecting the indication of the second test signal into the EUT using a QMU, Further processing the second test signal using the EUT, Communicating the processed second test signal from the EUT to the first device via a network; Further processing the communicated and processed second test signal to recover the display of the second test signal using the first device, and And objectively analyzing the perceptual quality of the recovered second test signal by comparing the recovered second test signal with the indication of the previously stored second test signal. . 14. The method of claim 13, wherein the analysis results of the quality of the recovered first test signal and the recovered second test signal are separated and stored, and are separated from the time stamps for later correlation and further processing of the stored results. Associating each memorized result with the method. The apparatus of claim 1, wherein the first device comprises an equipment under test (EUT), and the second device is operatively coupled to and receives a test signal for injecting a signal into the EUT for transmission as a test signal. A quality measuring unit (QMU) coupled to the network for Injecting an indication of a test signal into the EUT using QUM, Further processing the test signal using the EUT, and And transmitting the processed test signal to the QMU via a network. Analyzing the quality of the recovered test signal further comprises processing the received and processed signals to achieve a recovered indication of the test signal, and recovering the recovered test signal by comparing the recovered test signal with a previously stored indication of the test signal. And analyzing the quality of the tested test signal. A quality measuring device for measuring the quality of a signal transmitted to an equipment under test (EUT) via a communication network. Communicate with the EUT, Analyze the signal received and recovered by the EUT to identify the recovered test signal, And an objective analysis of the perceptual quality of the identified and recovered test signal. 17. The apparatus of claim 16, further comprising: analyzing at least one type of a voice signal, an auditory signal, a visual signal, and at least one type of the test signal selected from the group consisting of multi-media signals. 18. The apparatus of claim 17, further configured to display an indication of the quality of the identified and recovered test signal. 18. The apparatus of claim 17, further configured to be synchronized with a test signal recovered by an EUT to identify the recovered test signal. 18. The apparatus of claim 17, further configured to analyze the quality of the identified and recovered test signal while the processed test signal is received by an EUT. 18. The apparatus of claim 17, further comprising a local memory, wherein the quality measuring device is further configured to store the identified and recovered test signal in the local memory and to analyze the identified and recovered test signal after storage. Characterized in that the quality measurement of the transmitted signal. 22. The apparatus of claim 21, wherein the apparatus for measuring quality is further configured to disconnect an EUT from a network prior to analyzing the identified and recovered test signal. 18. The apparatus of claim 17, further comprising selecting a quality determination method for objectively analyzing perceptual quality of the identified and recovered test signals according to characteristics of the network. 21. The apparatus of claim 20, further configured to communicate with a radiotelephone that is an EUT. 21. The apparatus of claim 20, further comprising a local reference data storage unit (LRDSU) having an indication of the stored test signal, wherein the EUT is further configured to communicate with the processed indication of the test signal via a network. Device for measuring the quality of transmitted signals. 27. The apparatus of claim 25, further configured to record an indication of an objective perceptual quality of the recovered and identified test signal and to timestamp the indication of the recorded quality. 27. The method of claim 25, further comprising receiving an indication of the test signal communicated by the EUT via a network under control of the quality measuring device, and objectively determining the perceived quality of the test signal communicated to the quality measuring device via a network. Quality measuring device of the transmitted signal, characterized in that further configured to analyze. A system for measuring the perceptual quality of at least one of an audio signal, an audio signal, an auditory-visual signal, and a multimedia signal of a communication device in an operating environment. A transmission device configured to transmit an audio signal, an audio signal, an audio-visual signal, and a processed test signal, which is at least one of multiple signals, via a communication network; A receiver configured to receive a processed test signal via a network and to process the processed test signal to recover the display of the test signal; Perceptual quality, characterized in that it comprises a quality measuring unit (QMU) coupled to the receiver and configured to analyze the perceptual quality of the recovered test signal by comparing the recovered test signal with an indication of a previously stored test signal. Measuring system. 29. The apparatus of claim 28, further comprising a remote data storage unit (RRDSU) operatively coupled to the network for communicating the processed test signal, wherein the QMU is configured to analyze the quality of the recovered test signal. A system for measuring perceptual quality, characterized by determining prior to receiving a test signal. 30. The system of claim 29 wherein the QMU has a display configured to display an indication of the quality of the received test signal.