Hot news:

If you find a spelling error, please select an incorrect text and press Ctrl+Enter. Thank you!

RUSSIAN

MSU Video Quality Measurement Tool

MSU Graphics & Media Lab (Video Group)

Projects, ideas: Dr. Dmitriy Vatolin
Implementation: Alexey Moskvin, Oleg Petrov, Sergey Putilin, Sergey Grishin, Arsaev Marat, Georgy Osipov


Basic Information About VQMT



What's new in MSU VQMT 12?


Download FREE non-commercial version with limitations or Buy PRO

Special offer for recent customers:
50% off! Know more...

Brief demonstration of VQMT 12 main features
VQMT 12 changes in metrics
PSNRSingle metric instead of 4 ones. Added "Total PSNR" average value. Added "RGB" and "YUV" color spaces. Added Legacy mode
SSIM fastAccuracy increased (error could be in 3'nd decimal place for FullHD). Removed input aligning (accuracy for unaligned inputs). Multithread work**. Consider correct range, legacy mode
SSIM preciseRemoved input aligning***. Multithread work**. Consider correct range, legacy mode
SSIM OpenCLConsider correct input range.
SSIM CUDARemoved input aligning***. VQMT 11.2: aligning reduced to 8 as in OpenCL metrics
SSIM GPU IDMinimal changes
MSSSIM fastAccuracy increased (error could be in 3'nd decimal place for FullHD). Removed input aligning (accuracy for unaligned inputs). Multithread work**. Consider correct range, legacy mode
MSSSIM preciseRemoved input aligning***. Multithread work**. Consider correct range, legacy mode
MSSSIM OpenCLConsider correct input range.
MSSSIM CUDARemoved input aligning***
3SSIMRemoved input aligning***. Multithread work. Consider correct range, legacy mode
3SSIM OpenCLConsider correct input range.
3SSIM CUDARemoved input aligning***
VQMRemoved input aligning***. optimized (refactor, work without conversion)
BlockingRemoved input aligning***. optimized (refactor, work without conversion). Accuracy increased, legacy mode
Blurring DeltaOptimized (refactor, work without conversion). Consider correct range, legacy mode
Blurring SigmaConsider correct range, legacy mode
DeltaOptimized (work without conversion). Value range changed: 0..1. 0..255 (or 0..100 for L) in legacy mode
MSADOptimized (work without conversion). Value range changed: 0..1. 0..255 (or 0..100 for L) in legacy mode
MSEOptimized (work without conversion). Value range changed: 0..1. 0..255 (or 0..100 for L) in legacy mode
NIQEBug fixes. accuracy increased (mean calculation logic changed). Implemented visualization. Added “NIQE Mean” average values. Common mean is real mean as always. Multithread work**
VMAFMultithreaded work**
Brightness flickering*Bugs fixed (VQMT 11.2)
Brightness Independent PSNRBugs fixed (VQMT 11.2)
Drop FramePlugin metric*
Noise EstimationPlugin metric*
Scene Change DetectorPlugin metric*
All metricsSince the interpretation of the input has changed, the values of all metrics can change, and they depends on VQMT settings (mainly - Sample conversion). Also, VQMT is generally optimized.
* in plug-ins metrics:only plug-ins of API 11 (for VQMT 11) is supported in VQMT 12.
** multithreaded workmultithreaded work could increase memory consumption. You should set low memory parallelism if memory consumption is unsufficient
*** input aligningBefore VQMT 12 inputs were expanded to multiple of 8 or 16 (dependent on metrics). It could reduce accuracy for not-aligned metrics. In VQMT 11.2 we set aligning to 8 instead of 16 for CUDA metrics. In VQMT 12 we totally removed aligning.

Brief Description


VQMT New main window features

MSU Video Quality Measurement Tool (VQMT) is a program for objective video quality assessment. It provides functionality for both full-reference (two videos are examined) and single-reference (one video is analyzed) comparisons. Basic scheme of the tool is illustrated on below.

VQMT input and output VQMT input and output

Program allows to answer the next questions:

  1. One codec has more blurring than another. On what frames? Average mark? *
  2. One codec has more blocking than another. On what frames? Average mark? *
  3. One codec has lower quality than another. On what frames? Average mark in PSNR, VQM, SSIM? *
* Corresponding "bad frames" are saved automatically as TIFF files.

Main Features

Feature FREE version PRO version
Flexible Functionality

The tool provides functionality for calculating:

  • metric values for each frame
  • average value for sequence
  • metric values for specific color component

GPU and other computing devices

The tool supports performing some metrics on CUDA compatible devices (nVIDIA video cards) and all other OpenCL compatible devices (wide set of video cards, CPUs, multicore systems)

Objective Metrics Support
  • PSNR
  • SSIM (2 variations, GPU, OpenCL support)
  • MS-SSIM (2 variations, GPU, OpenCL support)
  • 3SSIM (GPU, OpenCL support)
  • stSSIM (temporary excluded)
  • VMAF
  • NIQE (no-reference metric)
  • VQM
  • Delta
  • MSAD
  • MSE
  • MSU Blurring Metric
  • MSU Blocking Metric
  • MSU Brightness Flicking Metric – opensource plugin
  • MSU Brightness Independent PSNR – opensource plugin
  • MSU Drop Frame Metric – opensource plugin
  • MSU Noise Estimation Metric
  • MSU Scene Change Detector
Video and Image Formats Support
  • Video files: *.AVI, *.DAT, *.DIVX, *.F4V, *.FLV, *.H265, *.M2TS, *.M2V, *.M4V, *.MKV, *.MOV, *.MP4, *.MPG, *.MTS, *.MXF, *.OGM, *.OGV, *.QT, *.TS, *.VOB, *.WMV, *.265, *.3G2, *.3GP, *.3GPP and others
  • AviSynth: *.AVS
  • RAW files: *.YUV (scientific), *.Y4M
  • Image files and image sequences: *.BMP, *.CUT, *.DDS, *.EXR, *.G3, *.GIF, *.HDR, *.ICO, *.IFF, *.LBM, *.J2K, *.J2C, *.JNG, *.JP2, *.JPG, *.JIF, *.JPEG, *.JPE, *.JXR, *.WDP, *.HDP, *.MNG, *.PCD, *.PCX, *.PCT, *.PICT, *.PIC, *.PNG, *.PBM, *.PGM, *.PPM, *.PSD, *.RAS, *.SGI, *.RGB, *.RGBA, *.BW, *.TGA, *.TARGA, *.TIF, *.TIFF, *.WAP, *.WBMP, *.WBM, *.WEBP, *.XBM, *.XPM
ROI Support (via masking)

Allows to calculate metrics in Regions of Interest (ROI). ROI should be specified by additonal provided video with mask for each frame.

Metrics Visualization

Video visualization file is generated for combination [metric, color component, reference]. Color and luminance of each pixel in visuzalization video is defined by metric value in that pixel (very useful for codec and filter debugging).

Results in CSV format

Metrics results and average values are saved in set of *.CSV (comma-separated values) file which can be opened everywhere.

Results in JSON format

All metric results and information about file opening are saved in single JSON file or stdout of command line utility.

Usage in companies

Free version license agreement allows only personal usage.

Comparative Analysis

Multiple references can be compared against the original simultaneously.

2 refs Unlim*
Professional Bit Depth Support

(over 8 bits per color component)

  • HDR video with 10, 12, 14, 16-bit RGB or YUV
  • Raw files with 9-16 bps or float samples
  • TIFF files with 1-16-bit integers or 32-bit floats
  • Whole set of PXM (PBM, PGM and PPM)
  • Batch Processing

    VQMT provides GUI and command-line (CL) interfaces. It is possible using CL-tool to calculate several metrics for a given set of color components simultaneously and for more files (in parallel), thus achieving significant speed advantage comparing to case of using GUI tool.

    Over HD Support

    Processing of video with resolution equal or higher than 1280x720: HD, FullHD, 4K, 8K video.

    Download FREE version Purchase PRO version
    multiple references allowed in command line interface

    Our plans

    VQMT back SDK

    Back SDK allows companies to integrate it's own products with VQMT metrics.

    Proceed to purchase SDK: Buy SDK

    VQMT SDK is provided as Linux shared library. VQMT back SDK compatible with C/C++, you can download interface here:

    Here you can also find SDK usage sample:

    Sample output:

    $ ./vqmt_sdktest ~/video/cinepack.yuv ~/video/loseless.yuv yuv420p 1280 720 all
    psnr:                      39.985
    psnr PSNR (256):           39.985
    psnr APSNR:                39.985
    psnr APSNR (256):          39.985
    ssim:                      0.960733
    ssim precise:              0.96347
    ssim GPU identical:        0.980512
    msssim:                    0.989333
    msssim precise:            0.993107
    3ssim:                     0.948871
    vqm:                       0.503724
    blocking:                  40.4037
    blurring:                  0.0115356
    blurring delta:            2.29336
    delta:                     0.000120949
    msad:                      0.873773
    mse:                       2.64237
    dfm:                       0
    bi_psnr:                   43.8853 2.65798
    niqe:                      8.24557
    scd:                       0
    vmaf:                      73.8748
    ne:                        0.00107096
    bfm:                       0
    

    Plots Examples


    These plots were built in MATLAB, based on measurement results from MSU VQMT.

    Results (1) Per-frame Y-PSNR for several codecs

    Results (2) Y-PSNR dependency vs bitrate and frame number

    Results (3) Average Y-PSNR for several codecs and bitrates

    Results (4) Colored surface of Y-PSNR vs frame number and bitrate

    Error (1) Example of the error in codec, detected with VQMT.
    Plots examples Another examples of plots built based on VQMT output.

    Comparison with Analogs



    Contributors


    Special thanks to following key contributors:
    Show more...
    Walt Disney Studios Intel Corporation Skype NASA Nvidia Corporation Apple Inc. Home Box Office, Inc. Microsoft Corporation Olympic Broadcasting Services 20th Century Fox Film Corporation Oracle Corporation Cisco Systems, Inc. Dolby Laboratories, Inc. Sony Computer Entertainment, Inc. Facebook, Inc. RealNetworks, Inc. QuickPlay Media, Inc. MediaTek, Inc. Fox Broadcasting Company NBCUniversal Ericsson Motorola Solutions, Inc. AVerMedia Technologies Inc. AMETEK, Inc. Sorenson Communications InterDigital Intuitive Surgical, Inc. TASER International, Inc. National Teleconsultants LLC Evertz Microsystems Limited Telestream, Inc University of Waterloo Simon Fraser University Wrocław University of Technology University of West Bohemia University of Vienna Fraunhofer-Institut für Digitale Medientechnologie IDMT Julius Maximilian University of Würzburg Federal University of Pará Hamburg University of Technology Analog Devices, Inc. Extron Electronics Lockheed Martin Exar Chips&Media, Inc. HDLogix, Inc. Comtronic Computer, Inc. Ittiam Systems VISLINK Whitney Medical Solutions Movidius Gaikai iMemories, Inc. Avaya Inc. Snell Ltd CEVA, Inc. Rockwell Collins, Inc. CDNetworks Elemental Technologies, Inc. Luxoft Smith Micro Software, Inc. BroadSoft Sky Deutschland AG Bell Media DirecTV Kaltura Netflix Inc. Vubiquity, Inc Telus Communications Beijing TL-Chinasoft Technology Co., Ltd Veset RGB Networks Burosch Audio-Video-Technik Telefónica, S.A. Allegro DVT Octasic Inc. Vlaamse Radio- en Televisieomroeporganisatie
    Studios
    Walt Disney Studios Home Box Office, Inc. Olympic Broadcasting Services 20th Century Fox Film Corporation Fox Broadcasting Company NBCUniversal
    Hardware & Devices
    Intel Corporation Nvidia Corporation Apple Inc. Cisco Systems, Inc. Sony Computer Entertainment, Inc. Dolby Laboratories, Inc. Ericsson Motorola Solutions, Inc. AMETEK, Inc. Evertz Microsystems Limited Telestream, Inc Analog Devices, Inc. Extron Electronics Lockheed Martin Exar Chips&Media, Inc. HDLogix, Inc. Comtronic Computer, Inc.
    Software & Solutions
    Skype Microsoft Corporation Oracle Corporation Facebook, Inc. QuickPlay Media, Inc. MediaTek, Inc. RealNetworks, Inc. AVerMedia Technologies Inc. Ittiam Systems Elemental Technologies, Inc. Luxoft Smith Micro Software, Inc. BroadSoft
    Education
    University of Waterloo Simon Fraser University Wrocław University of Technology University of West Bohemia University of Vienna Fraunhofer-Institut für Digitale Medientechnologie IDMT Julius Maximilian University of Würzburg Federal University of Pará Hamburg University of Technology
    Television & Media Industry
    Sky Deutschland AG Bell Media DirecTV Kaltura Netflix Inc. Vubiquity, Inc Telus Communications Beijing TL-Chinasoft Technology Co., Ltd Veset RGB Networks Burosch Audio-Video-Technik Telefónica, S.A. Allegro DVT Octasic Inc. Vlaamse Radio- en Televisieomroeporganisatie
    Others
    Sorenson Communications InterDigital Intuitive Surgical, Inc. TASER International, Inc. National Teleconsultants LLC VISLINK Whitney Medical Solutions Movidius Gaikai iMemories, Inc. Avaya Inc. Snell Ltd CEVA, Inc. Rockwell Collins, Inc. CDNetworks
    Show less

    Testimonials


    We have testimonials from the following big companies: Small companies: And universities:

    Changelog



    MSU Video Quality Measurement Tools



    Contacts


    e-mail: 


    Other Resources


    Video resources:

    Bookmark this page:   Add to Del.icio.us Add to Del.icio.us     Digg It Digg It     reddit reddit

     
    Last updated: 10-December-2019

    Search (Russian):
    Server size: 8069 files, 1215Mb (Server statistics)

    Project updated by
    Server Team and MSU Video Group


    Project sponsored by YUVsoft Corp.

    Project supported by MSU Graphics & Media Lab