KR100733261B1 - Objective Video Quality Evaluation System and Method Using Mixed Quality Index and Video Coding Apparatus and Method Therefor - Google Patents

Abstract

The present invention relates to an objective video quality evaluation system and method using a mixed picture quality index for improving the quality evaluation performance of a receiver by performing a picture quality evaluation using a picture quality evaluation result of a receiver at a receiver.

An objective video quality evaluation system using the mixed picture quality index of the present invention is a video quality evaluation system including a video transmitter which enables a video receiver to perform video quality evaluation using the video quality evaluation data of the video transmitter. The transmitter includes a data mixing unit which generates mixed data by mixing the encoded data and the image quality evaluation data; And a transmitter for transmitting the mixed data generated by the data mixer to the video receiver. There is a technical feature to including.

Description

System and method for evaluating objective video quality using mixed video quality index and system and method for video encoding {sequence and embedded video quality measurement using embedded video quality scores, and encoding apparatuses and methods therefor}

1 is a conceptual diagram of a video quality evaluation system for explaining the present invention;

2a to 2d is a block diagram of a video transmitter according to an embodiment of a video quality evaluation system according to an embodiment of the present invention;

3 is a diagram illustrating an example of an image quality evaluation index on a time axis;

4 is a diagram illustrating an image deterioration when a transmission error occurs;

5 is an embodiment showing frame loss when a transmission error occurs;

6a to 6c is a block diagram of a video receiver according to an embodiment of a video quality evaluation system according to an embodiment of the present invention;

7 is a flowchart sequentially illustrating a transmitter processing procedure of a video quality estimation method according to an embodiment of the present invention;

8 is a flowchart sequentially illustrating a receiver processing procedure of a video quality estimation method according to an embodiment of the present invention;

9 is a block diagram of a video encoding apparatus according to an embodiment of the present invention;

10 is a flowchart sequentially illustrating a video encoding method according to an embodiment of the present invention.

<Description of Symbols for Main Parts of Drawings>

100: transmitter 110: original video input unit

120: encoder 130: decoder

140: image quality evaluation unit 150: data mixing unit

160: feature extraction unit 170: transmitter

200 receiver 210 receive data input unit

220: data separator 230: decoder

240: transmission error estimation unit 250: image quality evaluation unit

260: original image input unit 270: feature data input unit

The present invention relates to a system and method for evaluating an objective video quality using a mixed video quality index and a video encoding apparatus and method for the same. When the image quality is evaluated by the receiver, the image quality can be evaluated by using the quality index and transmission error information of the transmitter, and the objective image quality can be performed by using the information acquired by the encoder or the decoder. The present invention relates to an objective video quality estimation system and method using a mixed video quality index, and a video encoding apparatus and method for the same.

Video quality evaluation is a core technology necessary for video transmission quality evaluation as well as field verification of video codec and development of new compression coding technique. Especially, due to the development of digital technology and communication technology, video is digitized and transmitted through network. With the advent of technology, the importance of objective video quality evaluation in a video transmission system compressed with this digital technology method is emphasized more.

The objective image quality evaluation technology can be used not only for television but also for mobile phone video quality evaluation. It is a core technology necessary for verifying the performance of video codecs, developing new compression coding techniques, and evaluating transmission quality. This technology is very important in that it can be used for the development and performance evaluation of related equipment such as digital cameras.

Objective image quality evaluation methods include Full-Reference, Reduced-Reference, and No-Reference.They are relatively accurate, but the scope of use is limited due to the need for moving images. The reduction reference method also has the disadvantage of requiring additional bandwidth, and the non-reference method, which requires no information of the original image, has the disadvantage of being the most versatile but low in accuracy. In particular, the non-standard method is very wide in scope if the performance is excellent, it can be used for real-time image quality monitoring is applied to unidirectional communication, such as digital multimedia broadcasting (DMB).

On the other hand, in the existing objective image quality evaluation system, the image quality evaluation apparatus generally operated independently of the video codec in performing the image quality evaluation by the above-described objective image quality evaluation method. For example, in the existing image quality evaluation system, when the reference image and the deteriorated video are provided to the image quality evaluation device, the image quality evaluation device evaluates the image quality by a method such as evaluating the image quality of the degraded video.

However, in most cases, an encoder or decoder of a video codec may be linked with an objective image quality evaluation device, and the performance of the objective image quality method may be improved by using auxiliary information provided by an encoder or a decoder of the video codec. .

Accordingly, the present invention is to solve the above-mentioned disadvantages and problems of the prior art, and transmits the image quality index, which is the result of the image quality evaluation of the encoded video at the transmitting side, together with the encoded video data, and at the formula side, the transmitted image quality index Performance of the objective image quality evaluation unit is improved by using the receiver and the transmission error information for the image quality evaluation of the receiver side, and the performance of the objective image quality evaluation method is utilized by evaluating the auxiliary information provided by the encoder or the decoder of the video codec for the image quality evaluation. An object of the present invention is to provide an objective video quality estimation system and method using a mixed video quality index and a video encoding apparatus and method therefor.

The objective video quality evaluation system using the mixed picture quality index of the present invention for achieving the above object, the video quality including a video transmitter that allows the video receiver to perform the video quality evaluation using the video quality evaluation data of the video transmitter An evaluation system comprising: a data mixing unit for generating mixed data by mixing encoded data and image quality evaluation data; And a transmitter for transmitting the mixed data generated by the data mixer to the video receiver. It includes.

In addition, the objective video quality evaluation system using the mixed picture quality index of the present invention for achieving the above object is a video quality evaluation system including a video receiver for performing a video quality evaluation using the video quality evaluation data of the video transmitter, The video receiver may further include: a reception data input unit configured to receive reception data, which is a mixture of data encoded through a network and image quality evaluation data, from the video transmitter; A data separation unit which receives the received data from the received data input unit and separates the encoded data into image quality evaluation data; A decoder which receives the encoded data from the data separator and decodes the received video to generate a received video; A transmission error estimating unit configured to estimate the transmission error by receiving the reception data from the reception data input unit; An image quality evaluation unit which receives image quality evaluation data from the data separation unit, receives transmission error information from the transmission error estimating unit, and evaluates image quality; It includes.

In addition, the objective video quality evaluation method using the mixed picture quality index of the present invention for achieving the above object is encoded, in order to enable the video receiver to perform video quality evaluation using the video quality evaluation data of the video transmitter. A moving picture quality estimation method in a moving picture transmitter including a data mixing unit for mixing data and picture quality evaluation data and transmitting the same to a video receiver, wherein the data mixing unit mixes the encoded data and the picture quality evaluation data to output mixed data; step; Transmitting the mixed data to a video receiver; It includes.

In order to achieve the above object, the objective video quality evaluation method using the mixed video quality index of the present invention comprises performing an image quality evaluation using a moving image input unit, an encoder, a decoder, and the decoded video data. In the video transmitter which consists of a picture quality evaluation unit and a data mixing unit for mixing the encoded data and the picture quality evaluation data to transmit to the video receiver, the video receiver to perform a video quality evaluation using the picture quality evaluation data of the video transmitter A moving picture quality estimation method comprising: a first step of inputting a moving picture to the encoder by the moving picture input unit; A second step of transmitting, by the encoder, the encoded moving image to the decoder and the data mixer; A third step of the decoder decoding the encoded video data and transferring the decoded video to the image quality evaluator; A fourth step of performing, by the image quality evaluation unit, image quality evaluation using the decoded video and transferring image quality evaluation data to the data mixing unit; A fifth step of mixing, by the data mixing unit, the encoded data and the image quality evaluation data to a video receiver; It includes.

In addition, the objective video quality evaluation method using the mixed picture quality index of the present invention for achieving the above object, the received data input unit for receiving data from the video transmitter over a network, the data separating unit for separating the received data, the decoding unit, A video quality estimation method in a video receiver including a transmission error estimator and a video quality evaluator, which performs video quality evaluation using the video quality evaluation data of the video transmitter, wherein the received data input unit encodes the video signal from the video transmitter through a network. A first step of receiving data in which the combined data and the image quality evaluation data are mixed and transmitting the data separator and the transmission error estimator; A second step of separating the received data into encoded data and image quality evaluation data, transferring the separated encoded data to the decoder, and transmitting the image quality evaluation data to the image quality evaluation unit; A third step of the transmission error estimator estimating a transmission error using the received data and transferring the estimated transmission error information to the image quality evaluator; A fourth step of evaluating the image quality using the image quality evaluation data transferred from the data separator and the transmission error information transmitted from the transmission error estimator; It includes.

Details of the above object and technical configuration of the present invention and the resulting effects thereof will be more clearly understood from the following detailed description based on the accompanying drawings.

First, FIG. 1 is a conceptual diagram of a video quality evaluation system for explaining the present invention.

As shown, the image quality evaluation system of the present invention is composed of a video transmitter 100 and a video receiver 200, so that the receiver can perform the image quality evaluation using the image quality evaluation result of the transmitter. The transmitter 100 transmits the encoded video data to the video receiver 200 by mixing the video quality evaluation data. Here, the image quality evaluation data may be a numerical value such as an image quality index calculated according to the image quality evaluation.

Accordingly, the video receiver 200 receives the mixed data, separates the encoded data into the encoded data and the image quality evaluation data, and evaluates the image quality using the image quality evaluation data.

Detailed components of the video transmitter 100 and the video receiver 200 will be described with reference to FIGS. 2 to 6 as follows.

First, FIG. 2A to FIG. 2D are diagrams illustrating the structure of a video transmitter according to an embodiment of a video quality evaluation system according to an embodiment of the present invention, and FIGS. 2A to 2C are electroconformance method (FIG. FIG. 2C illustrates a transmitter when the non-standard method (FIG. 2C) is applied, and FIG. 2D illustrates an embodiment of evaluating image quality by receiving encoded data and feature parameters from an external source without performing encoding and feature parameter extraction directly from a transmitter. It is shown.

As shown, FIGS. 2A to 2D show video transmitters 100 for evaluating encoded video quality at a transmitter and transmitting the evaluation result to the video receiver 200. FIG. It includes all of the data mixing unit 150 for mixing and transmitting.

In the present invention, the image quality evaluator 140 may evaluate the image quality by various methods such as full-reference, reduced-reference, and no-reference, and according to the image quality evaluation method, the transmitter 100 may further include some components.

As shown in FIG. 2A, in the case of applying the electroconductivity, which is a method of performing image quality evaluation using a moving image, the transmitter 100 of the present invention includes a moving image input unit 110, an encoding unit 120, and a decoding unit. 130, the image quality evaluator 140, the data mixer 150, and the transmitter 170.

The image quality evaluation unit 140 receives the moving image data from the moving image input unit 110, and processes the video, that is, the moving image decoded by the decoding unit 130 and the moving image which is a reference video input from the moving image input unit 110. Perform the quality evaluation using all of them. At this time, by receiving the auxiliary information from the encoder 120 can improve the performance of the image quality evaluation.

The moving image input unit 110 is for inputting a moving image (source video), and may be an input device that receives a moving image from an external device, or may be a predetermined storage device in which the moving image is stored in advance.

The encoder 120 encodes (compresses) the original image and provides encoded video data to the decoder 130 and the data mixer 150. In addition, in the present invention, in order to use auxiliary information that can be provided by the codec in the image quality evaluation of the video, as shown by the dotted line, the encoder 120 is helpful for image quality evaluation of information acquired in the encoding process. The auxiliary information may be provided to the image quality evaluator 140.

At this time, the auxiliary information includes codec type information, bit rate, frames per second (FPS), blocking degree indicating discontinuity between adjacent blocks, motion amount, and residual error. Information, and other useful information for image quality evaluation.

The relationship between the auxiliary information of the encoder 120 and the image quality evaluation is as follows.

First, since the image quality at the same bit rate varies greatly depending on the codec type, the information about the codec is very useful for the image quality evaluation. In addition, the bit rate is information that enables the overall prediction of the image quality, and the bit rate information allocated differently according to the frame may be important information for locally changing image quality prediction.

In addition, since the motion is awkwardly perceived when the number of frames per second is low, the number of frames per second becomes an important factor of image quality deterioration. When the frame number is lower than the frame number of the original image, the positional information on the time axis of the encoded frame may also be important information for image quality evaluation, and thus this information is also transmitted to the image quality evaluation unit 140.

In addition, the information about the degree of blocking is a very important factor of cognitive quality deterioration, but it is not easy to calculate the degree of blocking in a deteriorated image. Especially, in the latest codec, it is very difficult to find the block boundary due to the variable block size.

However, since the encoder 120 can accurately know the position of the block and can accurately predict the degree of blocking, the encoder 120 transmits information on the expected degree of blocking to the image quality evaluator 140 so as to accurately predict the image quality. To make it possible. If a de-blocking technique is used, information on this is also transmitted to the image quality evaluator 140.

In addition, even if each video has the same bit rate, a large difference occurs in the quality of the video encoded according to the content of the video. In other words, the video having low motion provides relatively good image quality even at a low bit rate. However, even with a high bit rate, the motion degree causes a deterioration of the video quality. Therefore, the information on the degree of movement is also transmitted to the image quality evaluator 140.

In addition, the encoder 120 may acquire accurate information about the residual error of each block after encoding, and may calculate an error for each frequency domain. Such information is very useful for predicting a blocking degree at a low bit rate or for predicting a fine error at a high bit rate. Accordingly, the encoder 120 may provide information about a residual error including a residual error for each frequency. 140).

Next, the decoder 130 decodes the encoded video data and transmits the decoded video data to the image quality evaluator 140.

The image quality evaluator 140 predicts the encoded video quality by using the moving image and the moving image received from the decoder 130, and outputs the image quality evaluation data (quality index) as a result. To pass.

, 비트율을

, 초당 프레임수를

, 인접한 블록 사이의 불연속성을 나타내는 블록킹(Blocking) 정도를

, 움직임 정도(Motion Amount)를

, 잔여 오차를

라고 하였을 때, 화질지수는 다음과 같이 수정될 수 있다.In this case, the image quality evaluator 140 may more accurately predict the encoded video quality by further using various auxiliary information transmitted from the encoder 120. For example, the codec type parameter

, Bitrate

, Frames per second

, The degree of blocking that represents discontinuity between adjacent blocks.

, The motion amount

, Residual error

In this case, the image quality index may be modified as follows.

은 원동영상과 복호화된 동영상에서 구한 화질지수,

는 파라미터

에 대한 함수,

는 보조정보를 이용하여 구한 수정된 화질지수다. 각 파라미터에 대한 가장 간단한 함수의 형태로 선형함수를 사용할 수 있으며, 이 때

는 다음과 같이 구할 수 있다.here

Is the quality index obtained from the original and decoded video,

Is a parameter

Function for,

Is the modified picture quality index obtained using the auxiliary information. You can use linear functions in the form of the simplest function for each parameter.

Can be obtained as

를 곱한 형태로 주어지며, 가중치를 학습데이터를 사용하여 최적화할 수 있다.Weighting each parameter

It is given as the product of, and the weight can be optimized using the training data.

The data mixer 150 mixes the encoded video data input from the encoder 120 with the image quality evaluation data input from the image quality evaluator 140 and transmits the image data to the transmitter 170. It transmits to the video receiver 200 through a network.

When encoding data and image quality evaluation data are mixed, they may be mixed using a special protocol, or may be mixed using a watermark technique or the like. With the Wittermark technique, the picture quality index can be mixed while maintaining compatibility with existing systems. The method of inserting additional information (quality index in the present invention) by using a watermark technique on existing data is not described in detail because various methods are introduced in many existing documents. For example, the picture quality index may be inserted in a least significant bit.

3 illustrates an example of an image quality evaluation index on a time axis, and as illustrated, a video quality index generally changes slowly. The picture quality index can be expressed relatively accurately by about 1 byte, and the picture quality index is enough for most applications when transmitting about once every 0.5-2 seconds. That is, the image quality index can be transmitted using about 1-2 bytes per second, and the image quality index of the transmitted video can be faithfully sent to the receiving side.

In addition, the amount of data required every few bytes is very small compared to the size of the compressed video data. In general, the size of the compressed video data is 10kbyte-1Mbyte per second, and the amount of additional data required to transmit the quality index is only 0.01% -0.0001%. Data of this size can be added to compressed video data using a watermark technique in consideration of compatibility.

According to the present invention, the receiver receives compressed video data mixed with the picture quality indexes transmitted by the transmitter, and evaluates the picture quality of the received video. If no transmission error occurs during transmission, the image quality of the received video is equal to the mixed image quality index. If a transmission error occurs, the receiver calculates the video quality of the received video by considering the mixed video quality index and the video quality degradation caused by the transmission error.

4 illustrates a deteriorated frame when a transmission error occurs. As shown in FIG. 4, the second to fourth frames represent frames deteriorated due to a transmission error. In this case, the received image quality becomes worse than the mixed image quality index, and the receiver calculates the image quality index from the mixed image quality index in consideration of the image quality degradation caused by the transmission error. For example, assuming that the higher the quality index, the better the image quality, the receiver image quality can be calculated as follows.

는 수신측의 화질지수,

는 송 신측에서 부호화후의 화질지수,

는 전송에러로 인한 화질 열화값으로 양수이다.here

Is the quality index of the receiver,

Is the quality index after encoding on the transmitting side,

Is a positive value of image quality deterioration due to a transmission error.

In addition, FIG. 5 shows an example in which frames are lost due to packet loss. As shown in FIG. 5, frames from the second frame to the fourth frame are lost frames due to packet loss.

As described above, in consideration of a case in which a frame loss due to packet loss occurs, similarity between previous frames and information on image content may be additionally mixed and transmitted as additional auxiliary information. When several frames are lost due to packet loss as shown in FIG. 5, most decoders repeatedly display the previous frame. At this time, in the case of a video having a little movement, the deterioration that a user feels decreases, and in the case of a lot of motion, the deterioration is severely felt.

Therefore, the performance of image quality prediction can be improved at the receiving side by adding a coefficient indicating the similarity and movement degree to the next frame to the current frame. This information can also be added using a few bytes of data per frame, and can be implemented using a watermark technique in consideration of compatibility. At this time, by receiving the auxiliary information from the encoder 120 can improve the performance of the image quality evaluation. Similarity with the previous frame can be calculated by obtaining the next and mean square error with the previous frame.

는

번째 프레임의

화소를 의미한다. 또한, 평균자승오차 대신 상관계수(correlation coefficient)와 같은 다른 함수를 사용하여 유사도를 측정할 수 있다. 움직임 정도는 움짐임 벡터(motion vector)의 크기, 움직임 보정 후 잔여 오차 등을 사용하여 정량화할 수 있다. 구체적인 계산방법은 본 발명에서 특정되지 않으므로, 여기에서는 상세하게 설명하지 않는다.Where M and N are the number of lines and columns

Is

Of the first frame

Means a pixel. In addition, the similarity may be measured using another function such as a correlation coefficient instead of the mean square error. The degree of motion can be quantified using the magnitude of the motion vector, residual error after motion compensation, and the like. Since a specific calculation method is not specified in the present invention, it will not be described in detail here.

On the other hand, when performing image quality evaluation by the reduction reference method as shown in Figure 2b, the transmitter 100 further includes a feature extraction unit (160).

That is, in FIG. 2B, the moving image input unit 110 inputs a moving image to the encoder 120 and the feature extractor 150, and the image quality evaluator 140 receives the feature parameter from the feature extractor 150. Image quality evaluation is performed using the encoded video and feature parameters.

In addition, in the case of performing image quality evaluation by the non-standard method as shown in FIG. 2C, the configuration of the transmitter 100 is the same as that of FIG. 2A, but unlike FIG. 2A, the moving image input unit 110 moves to the image quality evaluation unit 140. It does not provide the original image data, but merely inputs the original image data to the encoder 120. In particular, the image quality evaluation method using the non-standard method can be used when only the encoded video is provided without the original video in the transmitter.

Accordingly, the image quality evaluator 140 performs image quality evaluation only on the processed video by decoding input from the decoder 130 without any information about the reference video or the moving image. Here, the image quality evaluator 140 may use auxiliary information from the encoder 120 to evaluate the image quality, as indicated by a dotted line.

In general, the non-standard quality evaluation method is less accurate than the electric qualitative quality evaluation method and the reduction standard evaluation method. However, on the transmitting side, encoded data includes only codec degradation.

That is, unlike the receiving side, the encoded data on the transmitting side does not include a transmission error. Accordingly, even if the same non-standard method is used, the transmitting side can more accurately perform image quality evaluation. Accordingly, the non-standard method is MPEG-2 TS. It is used to estimate video quality by analyzing bit streams included in (moving picture experts group 2, transport stream).

On the other hand, in many cases, since there is no original video on the transmitting side such as a communication company, the communication company receives the encoded video from the content provider and transmits it to the end user. In this case, the content provider may generate the mixed data by mixing the quality evaluation data and the encoded data by using the above-described method and provide it to the telecommunications company.

If it is impossible for the content provider to generate mixed data by mixing the image quality evaluation data and the encoded data, the transmitter generates the image quality evaluation data by the reduction reference method or the no reference method, and then mixes the image quality evaluation data with the encoded data to the receiver. Can transmit

2D illustrates an embodiment in which the transmitter 100 does not directly perform encoding and feature parameter extraction. If the content provider provides the feature parameter for reducing reference image quality evaluation together with the encoded data, the transmitter 100 The coded data input unit 120-1 and the feature parameter input unit 160-1 are provided for image quality evaluation using the reduced reference method, and the encoder or the feature extractor in the transmitter of FIG. Do not.

That is, in the transmitter 100 of FIG. 2D, the feature parameter input unit 160-1 receives feature data from the outside and transmits the feature data to the image quality evaluator 140, and the encoded data input unit 120-1 is pre-coded from the outside. After receiving and decoding the video data, the decoded video is transmitted to the image quality evaluator 140. Accordingly, the image quality evaluator 140 receives only the input feature parameter without directly receiving the original image and evaluates the image quality by comparing the parameters. In this case as well, the decoding unit 130 may receive auxiliary information to improve performance of image quality evaluation.

If the content provider only provides the encoded data, as shown in FIG. 2C, the image quality evaluation data may be generated using the non-standard method. In this case, the transmitter has a configuration excluding only the feature parameter input unit 160-1 in FIG. 2D.

Next, FIGS. 6A to 6C are diagrams illustrating configuration of a video receiver according to an embodiment of a video quality evaluation system according to an embodiment of the present invention, which is encoded with quality evaluation data from the video transmitter 100 described with reference to FIGS. 2A to 2C. FIG. 6A illustrates a video receiver 200 that receives data mixed with video data and performs image quality evaluation using image quality evaluation data. FIG. 6A illustrates a non-standard method, FIG. 6B illustrates an electro-compliance method, and FIG. 6C illustrates a reduced reference method. It is applied.

First, as shown in FIG. 6A, the video receiver 200 includes a received data input unit 210, a data separator 220, a decoder 230, a transmission error estimator 240, and an image quality evaluator 250. It includes.

The reception data input unit 210 receives data mixed with the video data encoded by the video transmitter 100 and the image quality evaluation data through a network, and receives the mixed data from the data separator 220 and the transmission error estimator. Forward to 240.

The data separator 220 separates the received mixed data into encoded data and image quality evaluation data, and the separated encoded data is a decoder 230 that decodes the encoded video data to generate a received video. The image is transmitted to the image quality evaluator 250.

The transmission error estimator 240 estimates a transmission error using the received data, and transmits the transmission error information to the image quality evaluator 250.

The image quality evaluator 250 evaluates the image quality of the receiver using the image quality evaluation data received from the data separator 220 and the transmission error information received from the transmission error estimator 240.

If a transmission error does not occur during transmission, the quality of the received video is the same as the mixed image quality index. If a transmission error occurs, the receiver 200 receives the received image in consideration of the mixed image quality index and the image degradation caused by the transmission error. To calculate the quality of the recorded video.

That is, when a transmission error occurs and is in a deteriorated frame, the reception image quality becomes worse than the mixed image quality index, so that the receiver 200 adds an additional degradation index to the mixed image quality index in consideration of the image quality degradation caused by the transmission error. The received picture quality index can be calculated. For example, when a transmission error occurs, the quality of the received video may be calculated as follows.

는 수신측의 화질지수,

는 송신측에서 부호화후의 화질지수,

는 전송에러로 인한 화질 열화값으로 양수이다.here

Is the quality index of the receiver,

Is the quality index after encoding at the transmitting side,

Is a positive value of image quality deterioration due to a transmission error.

는 다양한 방법으로 계산할 수 있으며 일예로 다음과 같이 산출될 수 있다.

Can be calculated in various ways and can be calculated as follows.

은 비트 에러율(bit error rate)이고,

는 전송 에러로 인해 열화가 발생한 프레임 수이며,

는 가중치이다.here

Is the bit error rate,

Is the number of frames where degradation occurred due to transmission errors,

Is the weight.

In addition, in consideration of a case in which a frame loss due to packet loss occurs, additional auxiliary information may be additionally mixed and transmitted with similarity between previous frames and information on image content.

는 다음과 같이 계산할 수 있다.As described above with reference to FIG. 5, when some frames are lost due to packet loss, most decoders repeatedly display previous frames, and transmit information on the frame loss to the image quality evaluator, which is useful for image quality evaluation. Can be used as information. At this time, in the case of a video having a little movement, the deterioration that a user feels decreases, and in the case of a video which has a lot of movement, the deterioration becomes severe. Therefore, by adding coefficients indicating similarity and movement degree to the next frame to the current frame, the performance of image quality prediction can be improved at the receiving side. If frame loss occurs like this

Can be calculated as

은 비트 에러율(bit error rate),

는 전송 에러로 인해 열화가 발생한 프레임 수,

은 인접 프레임간의 유사도(similarity ratio)를 나타내는 것으로 상관계수, 차이 등을 사용할 수 있으며,

는 움직임 정도를 나타내고,

는 가중치이다. 가중치는 손실된 프레임 수에 따라 가변할 수 있으며, 가중치를 학습데이터를 사용하여 최적화할 수 있다. 만일, 전송에러로 인 한 도 4와 같은 블록에러 발생 시, 에러가 발생한 블록에러의 수를 고려하여

를 계산할 수 있다. 일 예로 다음과 같이 계산할 수 있다.here

Is the bit error rate,

Is the number of frames where degradation occurred due to transmission errors,

Denotes the similarity ratio between adjacent frames and can use correlation coefficient, difference, etc.

Indicates the degree of movement,

Is the weight. The weight may vary depending on the number of frames lost, and the weight may be optimized using the training data. If a block error as shown in FIG. 4 due to a transmission error occurs, the number of block errors in which an error occurs

Can be calculated. For example, it may be calculated as follows.

는 블록에러가 발생한 블록의 수를 나타낸다.here

Represents the number of blocks in which a block error occurred.

를 계산할 수 있다In the above, all of them use only linear functions, but in some cases,

Can be calculated

Meanwhile, the image quality evaluator 250 receives the decoded video from the decoder 230 to use the image quality evaluation, or further receives auxiliary information for image quality evaluation from the information obtained during the decoding process from the decoder 230. It can be used for image quality evaluation.

That is, the decoder 230 may generate the received video by decoding the received encoded video data, and transmit the received video to the image quality evaluator 250, and decode the information acquired in the decoding process to use the image quality evaluation. The unit 230 may transmit the auxiliary information for the image quality evaluation of the information acquired in the decoding process to the image quality evaluation unit 250, which is equally applied to FIGS. 6B and 6C.

The auxiliary information includes codec type information, bit rate, frames per second (FPS), position on a time axis of a frame, block position information, blocking degree indicating discontinuity between adjacent blocks, and motion. Motion Amount, transmission bit information for each frequency, residual error information, transmission error information (bit error information, packet loss information, delay information, jittering information, etc.), and the like. An example of using this additional information is as described above.

The relationship between the auxiliary information of the decoder 230 and the image quality evaluation is as follows.

First, as in the encoding process, since the image quality at the same bit rate varies greatly according to the codec type, the information about the codec is very useful for evaluating the image quality, and the bit rate is information that enables the overall prediction of the image quality. The differently allocated bit rate information may be important information for locally changing picture quality prediction.

In addition, since the motion is awkwardly perceived when the number of frames per second is low, the number of frames per second becomes an important factor of image quality deterioration. If the frame is coded lower than the number of frames of the original image, since the positional information on the time axis of the encoded frame may also be important information for image quality evaluation, this information is also transmitted to the image quality evaluator 250.

In addition, the information on the degree of blocking is a very important factor of cognitive quality deterioration, but it is not easy to calculate the degree of blocking in the deteriorated image, and in particular, in the latest codec, it is very difficult to find the block boundary because the block size is variable. However, since the decoder 230 may accurately know the position of the block and predict the blocking degree more accurately, the decoder 230 may provide the information about the expected blocking degree together with the block position information and the image quality evaluator 250. This allows accurate picture quality prediction.

In addition, even if each video has the same bit rate, a large difference occurs in the quality of the video encoded according to the content of the video. In other words, the video having low motion provides relatively good image quality even at a low bit rate. However, even with a high bit rate, the motion degree causes a deterioration of the video quality.

In general, calculating the residual error for each frequency is possible only when the original image is provided. However, even if there is no reference moving image, the decoder 230 may calculate the energy transmitted for each frequency, and transmit such information to the image quality evaluator 250 to predict the residual error for each frequency, thereby improving image quality evaluation performance. You can.

In addition, transmission error information such as bit error, packet loss, delay, and jitter is an important factor in image quality evaluation, and the decoder 230 transmits the information about the transmission error to the image quality evaluator 250, thereby providing more accurate information. Enable picture quality evaluation.

Meanwhile, in the present invention, the image quality evaluator 250 may evaluate image quality by various methods such as full-reference and reduced-reference, in addition to the aforementioned no-reference method. According to the image quality evaluation method, the receiver 200 may further include some components.

That is, in the case of applying the electro-observation method, as illustrated in FIG. 6B, the receiver 200 further includes a moving image input unit 260, and the image quality evaluation unit 250 receives a moving image from the moving image input unit 260.

Therefore, the image quality evaluator 250 of FIG. 6B performs image quality evaluation by using the received video, the moving image, the image quality evaluation index, transmission error information, and the like. In addition, the decoder 230 may transmit auxiliary information for image quality evaluation of the information acquired during the decoding process to the image quality evaluator 250, and the image quality evaluator 250 may further use the image quality evaluator to perform image quality evaluation. Can be.

6C, the receiver 200 further receives a feature data input unit 270 that receives a feature extracted for image quality evaluation from an original image and transmits it to the image quality evaluator 250. Include. Feature data may be transmitted using the same channel or may be transmitted using other channels.

Accordingly, the image quality evaluator 250 performs image quality evaluation using the decoded video, input feature data, received video quality data, transmission error information, and the like. In addition, the decoder 230 may transmit auxiliary information for image quality evaluation, which is acquired in the decoding process, to the image quality evaluator 250, and the image quality evaluator 250 may further use the image quality evaluation to perform image quality evaluation. Can be.

Next, FIG. 7 is a flowchart sequentially illustrating a transmitter processing procedure of a video quality estimation method according to an embodiment of the present invention, and illustrates a process of evaluating image quality and transmitting data by a non-standard method in a transmitter.

As shown, when the moving image input unit 110 of the transmitter 100 inputs a moving image to the encoding unit 120 (S101), the encoding unit 120 receives the moving image received from the moving image input unit 110. The encoding is transmitted to the decoder 130 and the data mixer 150 (S102).

The decoder 130 transmits the decoded video to the image quality evaluator 140 (S103), and the image quality evaluator 140 uses the received decoded video to determine the image quality of the receiver. The image quality evaluation data is transmitted to the data mixing unit 150 (S104).

Next, the data mixer 150 mixes the encoded video data received in step S102 and the image quality evaluation data received in step S104, and the transmitter transmits the mixed data to the video receiver 200 through a network. (S105).

On the other hand, the embodiment shows the image quality evaluation process by the non-standard method performed in the transmitter 100 of Figure 2c, when performing the image quality evaluation by the electro-observation method as shown in Figure 2a, the original image input unit ( 110 inputs a moving image to the encoder 120 and the image quality evaluator 140. In step S104, the image quality evaluator 140 performs image quality evaluation using both the encoded video and the moving image.

In addition, when performing image quality evaluation by the reduction reference method as shown in Figure 2b, the moving image input unit 110 inputs the moving image to the encoder 120 and the feature extractor 150, the image quality in step S104 The evaluator 140 receives the feature parameter from the feature extractor 150 and performs image quality evaluation using the encoded video and the feature parameter. If the moving picture does not exist in the transmitter and only the encoded video and the feature parameters are input as shown in FIG. 2D, the image quality may be evaluated by the reduction reference method.

In addition, irrespective of the electric compliance method, the non-standard method, and the reduced reference method, the image quality evaluator 140 may further receive auxiliary information from the encoder 120 before step S104, and use the auxiliary information when the image quality is evaluated in step S104. Image quality evaluation can be performed.

Next, FIG. 8 is a flowchart sequentially illustrating a receiver processing procedure of a video quality estimation method according to an embodiment of the present invention, and illustrates a process of evaluating image quality by applying a non-standard method in a receiver.

As shown in the drawing, the reception data input unit 210 of the video receiver 200 receives mixed data received through a network, that is, data in which encoded video data and image quality evaluation data are mixed. And it transmits to the transmission error estimation unit 240 (S201).

The data separating unit 220 separates the received mixed data into encoded data and image quality evaluation data, and transmits the separated encoded data to the decoder 230 and the image quality evaluation data to the image quality evaluator 250 (S202). .

Next, the transmission error estimator 240 measures the transmission error and transmits the transmission error information to the image quality evaluator 250 (S203), and the image quality evaluator 250 transmits the transmission error information and the decoder 230. The image quality evaluation is performed using the decoded video decoded in step S and the image quality evaluation data received in step S202 (S204).

On the other hand, the embodiment shows the image quality evaluation process by the non-standard method, when performing the image quality evaluation by the electro-observation as shown in Figure 6b, in step S204 the image quality evaluation unit 250 is decoded video and moving image input unit Image quality evaluation is performed using the original image input from 240.

In addition, when performing image quality evaluation by the reduction reference method as shown in Figure 6c, in step S204 the image quality evaluation unit 250 receives the feature parameters from the feature extractor 250 to use the decoded video and feature parameters Image quality evaluation.

The image quality evaluator 250 further receives the decoded video and the auxiliary information from the decoder 230 between steps S202 and S204, and further evaluates the image quality using the decoded video and the auxiliary information in step S204. Can be done.

In addition, since one of the important purposes of the image quality evaluation of the receiving side is the image quality monitoring of the transmitting side, the image quality evaluator 250 preferably provides the image quality evaluation result to the transmitter 100.

9 and 10 are diagrams for describing a video encoding apparatus and a video encoding method of the present invention.

In the video image quality evaluation system described above, the video transmitter 100 mixes the image quality evaluation data and the encoded data and transmits the image quality evaluation data and the encoded data to the video receiver 100 so that the video receiver 200 may perform the image quality evaluation using the mixed image quality index. do.

In this case, the video transmitter 100 generates both mixed data and transmits the mixed data to the video receiver 200. However, the mixed data is generated by a separate video encoding apparatus, and the video transmitter only transmits the mixed data. It can only play the role of transmitting to the receiver.

In this case, the video encoding apparatus 300 that generates the mixed data includes the original video input unit 310, the encoder 320, the decoder 330, the image quality evaluator 340, and the data mixer as shown in FIG. 9. 350, the role of each component is the same as in FIG. 2A described above.

That is, the video encoding apparatus 300 of FIG. 9 is configured to remove only the transmitter 170 from the transmitter 100 of the video quality evaluation system described above with reference to FIG. 2A, and the image quality estimation unit 340 of the video encoding apparatus of the present invention. Since the image quality may be evaluated using the electro-computation method, the reduction reference method, and the non-standard method, the video encoding apparatus of the present invention may take the configuration excluding the transmitter 170 in FIGS. 2B and 2C, respectively.

Similarly, since the flowchart of FIG. 10 illustrates a video encoding process performed by the video encoding apparatus 300 of the present invention, the flowchart of FIG. 10 is almost the same as the video transmission process of the video transmitter described above with reference to FIG. 7.

Specifically, steps S301 to S304 are the same as steps S101 to S104 of FIG. 7, and in step S305, the data mixer 350 of the video encoding apparatus 300 of the present invention encodes the encoded video received in step S302. The data and the image quality evaluation data received in step S304 are mixed. That is, since the video encoding apparatus 300 of the present invention does not include a transmitter, the mixed video is not transmitted to the video receiver.

Meanwhile, as described above in the description of the image quality evaluator 340 of the video encoding apparatus 300, the image quality evaluator 340 evaluates image quality by applying various methods such as a non-standard method, a reduced reference method, and an all reference method. Can be done.

As those skilled in the art to which the present invention pertains may implement the present invention in other specific forms without changing the technical spirit or essential features, the embodiments described above should be understood as illustrative and not restrictive in all aspects. Should be. The scope of the present invention is shown by the following claims rather than the detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalents should be construed as being included in the scope of the present invention. do.

Therefore, according to the objective video quality estimation system and method using the mixed video quality index of the present invention and the video encoding apparatus and method therefor, the video quality index on the receiving side is mixed with the encoded data and transmitted, thereby transmitting the quality. It is to facilitate the evaluation, and the objective image quality evaluation performance can be improved by utilizing the auxiliary information provided by the encoder or the decoder of the video codec in the image quality evaluation.

Claims (30)

delete A video quality evaluation system including a video transmitter that enables a video receiver to perform video quality evaluation using video quality evaluation data of the video transmitter. The video transmitter, A moving image input unit for inputting a moving image; An encoder which encodes the input original image; A decoder which decodes video data encoded by the encoder; An image quality evaluation unit configured to perform image quality evaluation on the decoded video and output image quality evaluation data; A data mixing unit which generates mixed data by mixing the encoded data and the image quality evaluation data; And A transmitter for transmitting the mixed data generated by the data mixer to the video receiver; Objective video quality evaluation system using a mixed picture quality index, characterized in that it comprises a. A video quality evaluation system including a video transmitter that enables a video receiver to perform video quality evaluation using video quality evaluation data of the video transmitter. The video transmitter, A moving image input unit for inputting a moving image into an encoder and an image quality evaluation unit; An encoder which encodes the input original image; A decoder which decodes video data encoded by the encoder; An image quality evaluation unit configured to perform image quality evaluation using the original image input from the original image input unit and the decoded video and output image quality evaluation data; A data mixing unit which generates mixed data by mixing the encoded data and the image quality evaluation data; And A transmitter for transmitting the mixed data generated by the data mixer to the video receiver; Objective video quality evaluation system using a mixed picture quality index, characterized in that it comprises a. The method according to claim 2 or 3, The encoder transmits auxiliary information for image quality evaluation from information acquired in an encoding process to the image quality evaluation unit. The image quality evaluation unit, the objective video quality evaluation system using a mixed image quality index, characterized in that for performing the image quality further using the auxiliary information. The method of claim 4, wherein The auxiliary information, Codec type information, bit rate, frames per second (FPS), position on the time axis of the frame, blocking degree indicating discontinuity between adjacent blocks, motion amount, frequency residual Objective video quality evaluation system using a mixed picture quality index, characterized in that it comprises at least one or more of the error information. A video quality evaluation system including a video receiver for performing video quality evaluation using video quality evaluation data of a video transmitter. The video receiver, A reception data input unit configured to receive reception data of a mixture of encoded data and image quality evaluation data through a network from the video transmitter; A data separation unit which receives the received data from the received data input unit and separates the encoded data into image quality evaluation data; A decoder which receives the encoded data from the data separator and decodes the received video to generate a received video; A transmission error estimating unit configured to estimate the transmission error by receiving the reception data from the reception data input unit; An image quality evaluation unit which receives image quality evaluation data from the data separation unit, receives transmission error information from the transmission error estimating unit, and evaluates image quality; Objective video quality evaluation system using a mixed picture quality index, characterized in that it comprises a. The method of claim 6, The video receiver, Further comprising a feature parameter input unit for inputting a feature parameter extracted from the original image, And the image quality evaluation unit performs image quality evaluation further using the feature parameter. The method according to claim 6 or 7, The decoder transmits the decoded video to the image quality evaluator, And the image quality evaluator further performs image quality evaluation using the decoded video input from the decoder. The method of claim 8, The decoding unit transmits auxiliary information for image quality evaluation among information acquired in the decoding process to the image quality evaluation unit, The image quality evaluation unit, the objective video quality evaluation system using the mixed image quality index, characterized in that to perform the image quality evaluation by further using the auxiliary information transmitted from the decoder. The method of claim 9, The auxiliary information, Codec type information, bit rate, frames per second (FPS), position on the time axis of the frame, block position information, blocking degree indicating discontinuity between adjacent blocks, and motion amount And, at least one of transmission bit information for each frequency, residual error information for each frequency, and transmission error information. The method of claim 10, The transmission error information, An objective video quality evaluation system using a mixed picture quality index comprising at least one of bit error information, packet loss information, delay information, and jittering information. delete An original video input unit, an encoding unit, a decoding unit, an image quality evaluation unit which performs image quality evaluation using the decoded video data, and a data mixing unit which mixes the encoded data and image quality evaluation data and transmits them to a video receiver. A video quality estimation method in a video transmitter that allows a video receiver to perform video quality evaluation using video quality evaluation data of the video transmitter. A first step of inputting a moving image to the encoder by the moving image input unit; A second step of transmitting, by the encoder, the encoded moving image to the decoder and the data mixer; A third step of the decoder decoding the encoded video data and transferring the decoded video to the image quality evaluator; A fourth step of performing, by the image quality evaluation unit, image quality evaluation using the decoded video and transferring image quality evaluation data to the data mixing unit; And A fifth step of mixing, by the data mixing unit, the encoded data and the image quality evaluation data to a video receiver; Objective video quality evaluation method using a mixed picture quality index, characterized in that it comprises a. The method of claim 13, In the first step, The moving image input unit inputs a moving image to the encoder and the image quality evaluation unit, In the fourth step, And the image quality evaluator further performs image quality evaluation using the original image. The method according to claim 13 or 14, After the first step and before the fourth step, And transmitting, to the image quality evaluator, the encoder by using the encoding unit, auxiliary information for image quality evaluation, among the information acquired in the encoding process. In the fourth step, And the image quality evaluator further performs image quality evaluation using the supplementary information. The method of claim 15, The auxiliary information, Codec type information, bit rate, frames per second (FPS), position on the time axis of the frame, blocking degree indicating discontinuity between adjacent blocks, motion amount, frequency residual Objective video quality evaluation method using a mixed picture quality index, characterized in that it comprises at least one or more of the error information. Evaluating video quality using the image quality evaluation data of the video transmitter, including a reception data input unit for receiving data from a video transmitter through a network, a data separator for separating received data, a decoder, a transmission error estimator, and an image quality evaluation unit. A video quality evaluation method in a video receiver that performs A first step of adding the received input data transfer receives the data and the quality assessment data is encoded mixed data over the network from the video transmitter to the data separation unit and sending the error estimator; A second step of separating the received data into encoded data and image quality evaluation data, transferring the separated encoded data to the decoder, and transmitting the image quality evaluation data to the image quality evaluation unit; A third step of the transmission error estimator estimating a transmission error using the received data and transferring the estimated transmission error information to the image quality evaluator; And A fourth step of evaluating the image quality using the image quality evaluation data transferred from the data separator and the transmission error information transmitted from the transmission error estimator; Objective video quality evaluation method using a mixed picture quality index, characterized in that it comprises a. The method of claim 17, The video receiver, Further comprising an input unit for receiving a feature parameter extracted from the original image, In the fourth step, And the image quality evaluation unit performs image quality evaluation further using the feature parameter. The method of claim 17 or 18, After the second step and before the fourth step, The decoding unit further includes the step of decoding the encoded video input from the data separating unit and forwards the decoded video to the image quality evaluation unit, In the fourth step, And the image quality evaluator further performs image quality evaluation using the decoded video input from the decoder. The method of claim 19, In the step of transmitting the decoded video to the image quality evaluator, Transmitting auxiliary information for image quality evaluation among information acquired in the decoding process to the image quality evaluation unit, In the fourth step, The image quality evaluation unit, the objective video quality evaluation method using a mixed image quality index, characterized in that for performing the image quality evaluation using the additional information transmitted from the decoder. The method of claim 20, The auxiliary information, Codec type information, bit rate, frames per second (FPS), position on the time axis of the frame, block position information, blocking degree indicating discontinuity between adjacent blocks, and motion amount And, at least one of transmission bit information for each frequency, residual error information for each frequency, and transmission error information. The method of claim 21, The transmission error information, A method for evaluating objective video quality using a mixed picture quality index comprising at least one of bit error information, packet loss information, delay information, and jittering information. A video encoding apparatus for mixing video quality indexes to perform video quality evaluation using video quality evaluation data. A moving image input unit for inputting a moving image; An encoder which encodes the input moving image and outputs an encoded video; A decoder configured to decode the encoded video and output a decoded video; An image quality evaluation unit which performs image quality evaluation of the encoded video using the decoded video and outputs image quality evaluation data; And A data mixer which mixes and outputs encoded video data which is output data of the encoder and image quality evaluation data that is output data of the image quality evaluator; Video encoding apparatus comprising a. The method of claim 23, wherein The image quality evaluation unit, A video encoding apparatus, characterized in that the image quality evaluation is performed using any one of the non-standard method, the electric compliance method, and the reduced reference method. The method of claim 23, wherein The mixing unit, The video encoding apparatus, characterized by further mixing data representing the similarity with the previous frame. The method of claim 23 or 25, The mixing unit, A video encoding device, characterized by further mixing coefficients representing the degree of motion. A video encoding method in a video encoding apparatus including a moving image input unit, an encoding unit, a decoding unit, an image quality evaluation unit, and a data mixing unit to mix image quality evaluation indices so as to perform video quality evaluation using image quality evaluation data. A first step of inputting a moving image to the encoder by the moving image input unit; A second step of transmitting, by the encoder, the encoded moving image to the decoder and the data mixer; A third step of the decoder decoding the encoded video data and transferring the decoded video to the image quality evaluator; A fourth step of performing, by the image quality evaluation unit, image quality evaluation using the decoded video and transferring image quality evaluation data to the data mixing unit; And A fifth step of mixing, by the data mixing unit, the encoded data and image quality evaluation data; Video encoding method comprising a. The method of claim 27, In the fourth step, The image quality evaluation unit, The video encoding method of claim 1, wherein the image quality evaluation is performed using any one of the non-standard method, the electro-observation method, and the reduced reference method. The method of claim 27, In the fifth step, The mixing unit, The video encoding method, characterized by further mixing the data indicating the similarity with the previous frame. The method of claim 27 or 29, In the fifth step, The mixing unit, The video encoding method, characterized by further mixing the data indicating the degree of motion.

Images