VQMT: Video Quality Measurement Tool

Introduction

This software provides fast implementations of the following objective metrics:

  • PSNR: Peak Signal-to-Noise Ratio,
  • SSIM: Structural Similarity,
  • MS-SSIM: Multi-Scale Structural Similarity,
  • VIFp: Visual Information Fidelity, pixel domain version,
  • PSNR-HVS: Peak Signal-to-Noise Ratio taking into account Contrast Sensitivity Function (CSF),
  • PSNR-HVS-M: Peak Signal-to-Noise Ratio taking into account Contrast Sensitivity Function (CSF) and between-coefficient contrast masking of DCT basis functions.

In this software, the above metrics are implemented in OpenCV (C++) based on the original Matlab implementations provided by their developers. The source code of this software can be compiled on any platform and only requires the OpenCV library (core and imgproc modules). This software allows performing video quality assessment without using Matlab and shows better performance than Matlab in terms of run time.

Prerequisite

The OpenCV library needs to be installed to be able to compile this code. Only the core and imgproc modules are required.

This software was developed using OpenCV 2.4.0, but earlier/newer versions of OpenCV are most likely compatible since only low-level features, such as filtering and resizing, are used.

The OpenCV dlls are provided directly with the Windows binaries, such that the software can work directly without installing the OpenCV library.

Usage

VQMT.exe OriginalVideo ProcessedVideo Height Width NumberOfFrames ChromaFormat Output Metrics

  • OriginalVideo: the original video as raw YUV video file, progressively scanned, and 8 bits per sample
  • ProcessedVideo: the processed video as raw YUV video file, progressively scanned, and 8 bits per sample
  • Height: the height of the video
  • Width: the width of the video
  • NumberOfFrames: the number of frames to process
  • ChromaFormat: the chroma subsampling format. 0: YUV400, 1: YUV420, 2: YUV422, 3: YUV444
  • Output: the name of the output file(s)
  • Metrics: the list of metrics to use (PSNR, SSIM, MSSSIM, VIFP, PSNRHVS, PSNRHVSM)

Example:

VQMT.exe original.yuv processed.yuv 1088 1920 250 1 results PSNR SSIM MSSSIM VIFP

will create the following output files in CSV (comma-separated values) format:

  • results_pnsr.csv
  • results_ssim.csv
  • results_msssim.csv
  • results_vifp.csv

Notes:

  • SSIM comes for free when MSSSIM is computed (but you still need to specify it to get the output)
  • PSNRHVS and PSNRHVSM are always computed at the same time (but you still need to specify both to get the two outputs)
  • When using MSSSIM, the height and width of the video have to be multiple of 16
  • When using VIFP, the height and width of the video have to be multiple of 8

Changes

Changes in version 1.1 (since 1.0) on 30/3/13

  • Added support for large files (>2GB)
  • Added support for different chroma sampling formats (YUV400, YUV420, YUV422, and YUV444)

FAQ

Q: How do I install OpenCV?
A: Look at this tutorial.

Q: How do I configure Visual Studio 2010 to include the OpenCV library?
A: Look at this tutorial.

Q: How do I write code using OpenCV?
A: Look at this book.

Q: I have the following error when running the software: “The program can’t start because tbb.dll is missing from your computer. Try reinstalling the program to fix this problem”. What should I do?
A: Copy the missing dll file from the OpenCV folder, located in ‘C:\OpenCV2.X\build\common\tbb\’, corresponding to the architecture (32 vs 64 bits) and version of Visual Studio you are using.

Q: I have the following error when running the software: “The program can’t start because opencv_core240.dll (or opencv_imgproc240.dll) is missing from your computer. Try reinstalling the program to fix this problem”. What should I do?
A: Copy the missing dll file from the OpenCV folder, located in ‘C:\OpenCV2.4\build\x86\vc10\bin’ if you are running the 32bits version or ‘C:\OpenCV2.4\build\x64\vc10\bin’ for the 64bits version.

Q: I don’t get the exact same results with VQMT than with the original Matlab implementations. Is there a bug in the software?
A: The Matlab implementations usually use 64 bits double-precision floating-point numbers while most OpenCV functions use 32 bits single-precision floating-point numbers. This is probably where the difference comes from. You should roughly get the same precision up to the 5th decimal.

Q: Where can I find a Makefile for Linux?
A: Have a look here: https://github.com/Rolinh/VQMT (note that this project is not maintained by EPFL nor MMSPG).

Copyright

Permission is hereby granted, without written agreement and without license or royalty fees, to use, copy, modify, and distribute the software provided and its documentation for research purpose only, provided that this copyright notice and the original authors’ names appear on all copies and supporting documentation. The software provided may not be commercially distributed. In no event shall the Ecole Polytechnique Fédérale de Lausanne (EPFL) be liable to any party for direct, indirect, special, incidental, or consequential damages arising out of the use of the software and its documentation. The Ecole Polytechnique Fédérale de Lausanne (EPFL) specifically disclaims any warranties. The software provided hereunder is on an “as is” basis and the Ecole Polytechnique Fédérale de Lausanne (EPFL) has no obligation to provide maintenance, support, updates, enhancements, or modifications.

Download

Source code only

Microsoft Visual Studio 2010 project

Windows 32/64 bits binaries with dlls

Contact

For questions, comments, bug report, or integration of new objective metrics, please contact Philippe Hanhart ([email protected])

References

  • Z. Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600–612, April 2004.
  • Z. Wang, E.P. Simoncelli, and A.C. Bovik, “Multiscale structural similarity for image quality assessment,” in IEEE Asilomar Conference on Signals, Systems and Computers, November 2003, vol. 2, pp. 1398–1402.
  • H.R. Sheikh and A.C. Bovik, “Image information and visual quality,” IEEE Transactions on Image Processing, vol. 15, no. 2, pp. 430-444, February 2006.
  • K. Egiazarian, J. Astola, N. Ponomarenko, V. Lukin, F. Battisti, and M. Carli, “New full-reference quality metrics based on HVS,” in Proceedings of the Second International Workshop on Video Processing and Quality Metrics, 2006.
  • N. Ponomarenko, F. Silvestri, K. Egiazarian, M. Carli, J. Astola, and V. Lukin, “On between-coefficient contrast masking of DCT basis functions,” in Proceedings of the Third International Workshop on Video Processing and Quality Metrics for Consumer Electronics, January 2007.