Start page

IQM2 image quality measure

Introduction

Objective image quality evaluation plays an important role in many image and video processing techniques, such as compression, interpolation, image processing, and watermarking where evaluation method is based on image quality estimation. Quality of image can be evaluated using different measures. The best way to do that is by making a visual experiment under controlled conditions, in which human observers grade image quality. Such experiments are time consuming and costly. Much easier approach is to use some objective measure that evaluates the numerical error between the original image and distorted image. Every objective image quality measure has its aim to approximate the human quality perception (or human visual system, HVS) as much as possible, which means to correlate well with subjective measures (mean opinion score, MOS). Depending on the development type, objective measures can be based on bottom-up or top-down approaches. In bottom-up approach, underlying premise is that the sensitivities of the human visual system (HVS) are different for different aspects of the visual signal that it perceives. Unlike these models, topdown approach is not affected by assumptions about HVS models, but is motivated instead by the need to capture the loss of visual structure in the signal that the HVS hypothetically extracts for cognitive understanding. Some objective measures can be a combination of both approaches.

IQM2 short description

Original and degraded (grayscale) images are firstly transformed using steerable pyramid wavelet transform (SPWT) with K orientations and maximal number of scales M and on each scale modified SSIM measure is calculated, with contrast and structure terms only. It has been shown that the best results were obtained using kernel with 2 orientations and modified SSIM with block size 5x5 pixels. These results are presented below, and compared with 13 other full-reference objective measures: CWSSIM, IWPSNR, IWSSIM, MAD, MSE, MSSIM, NAE, NQM, SSIM, SSIMmod, VIF, VIFP, VSNR. They have been compared using 7 publicly available image databases: A57, CSIQ, LIVE, IVC, VCL@FER, TID and TOYAMA. Here, comparison was made using Spearman's and Kendall's correlation. MEAN is mean value across all databases, while WT_MEAN is weighted mean (database size is taken into account). It can be seen that best performing measures are IQM2, IWSSIM and MAD measures. MAD measure has highest correlation in 4 tested databases. IQM2 has highest weighted mean correlation, highest correlation in TID database (and statistically significant comparing with any other measure using F-test or Ansari-Bradley test), lower computation time than IWSSIM and significantly lower computation time than MAD. However, their correlation is still far from 1, especially in databases with more degradation types (TID). Table Timing gives calculation times for all objective measures in ms. Computer configuration which was used for calculating all objective measures: Intel Q6600 @2400 MHz, 4 GB RAM, Windows Vista 64 with Matlab program. Mean time was calculated for all degraded images in TID database (1700 images with resolution 512•384 pixels). Converting to grayscale and scaling images if needed were not taken in calculation time. MEX files were used where possible (for SPWT transform and MAD measure). For more details, please refer to the paper below.

Results


Spearman's correlation


CWSSIM IQM2 IWPSNR IWSSIM MAD MSE MSSIM NAE NQM SSIM SSIMmod VIF VIFP VSNR
A57 0.33148 0.83964 0.87619 0.87127 0.90139 0.61763 0.8415 0.56484 0.79778 0.80666 0.80662 0.62228 0.76854 0.93588
CSIQ 0.8412 0.93766 0.83106 0.92129 0.94665 0.8058 0.91364 0.75965 0.74116 0.87563 0.92839 0.91945 0.88068 0.81095
LIVE 0.9025 0.95064 0.93278 0.95665 0.96689 0.87556 0.95128 0.8367 0.9093 0.9479 0.94783 0.96315 0.96179 0.92713
IVC 0.85791 0.88169 0.89976 0.9125 0.91457 0.68844 0.898 0.60877 0.83431 0.90182 0.90285 0.89637 0.81091 0.79927
VCL@FER 0.83477 0.93497 0.9166 0.91633 0.90607 0.82465 0.92269 0.79464 0.94359 0.91125 0.91001 0.88665 0.89185 0.87261
TID 0.65859 0.88547 0.68234 0.85594 0.83401 0.5531 0.85418 0.32525 0.62359 0.77493 0.81771 0.74907 0.65389 0.70488
TOYAMA 0.81531 0.87288 0.8475 0.92024 0.93617 0.61319 0.88738 0.51649 0.8871 0.87938 0.87943 0.90767 0.84789 0.86082
MEAN 0.74882 0.90042 0.85518 0.90775 0.91511 0.7112 0.89552 0.62948 0.81955 0.87108 0.88469 0.84923 0.83079 0.84451
WT_MEAN 0.77266 0.91289 0.80586 0.9002 0.89826 0.70611 0.89553 0.58808 0.76153 0.85391 0.8813 0.85068 0.80153 0.801

Kendall's correlation


CWSSIM IQM2 IWPSNR IWSSIM MAD MSE MSSIM NAE NQM SSIM SSIMmod VIF VIFP VSNR
A57 0.23007 0.65804 0.6972 0.68462 0.72238 0.43007 0.64825 0.39231 0.59231 0.60629 0.6049 0.45944 0.56434 0.8035
CSIQ 0.64808 0.77846 0.65917 0.75287 0.79701 0.60836 0.7395 0.56044 0.56534 0.6907 0.75755 0.75373 0.69693 0.62475
LIVE 0.71818 0.80669 0.78003 0.81752 0.84213 0.68646 0.80445 0.63975 0.74309 0.79629 0.79603 0.82701 0.82498 0.76095
IVC 0.66963 0.70766 0.71652 0.73388 0.74061 0.52175 0.7203 0.44085 0.63372 0.72231 0.7262 0.71581 0.63077 0.60526
VCL@FER 0.63314 0.76862 0.73843 0.7372 0.72135 0.63614 0.74973 0.60074 0.78173 0.73315 0.73108 0.69244 0.70019 0.68278
TID 0.4783 0.70309 0.52547 0.66364 0.64451 0.40275 0.65685 0.23008 0.46 0.57676 0.61347 0.58605 0.49451 0.53451
TOYAMA 0.62329 0.68286 0.65077 0.75366 0.78229 0.44428 0.70286 0.36715 0.70488 0.69394 0.69394 0.7315 0.65868 0.67451
MEAN 0.57153 0.72935 0.68108 0.73477 0.75004 0.53283 0.71742 0.46162 0.64015 0.68849 0.70331 0.68085 0.65291 0.66947
WT_MEAN 0.58651 0.74425 0.64102 0.72568 0.7313 0.53248 0.71652 0.43468 0.59238 0.67068 0.69847 0.68671 0.63457 0.62455

Timing


CWSSIM IQM2 IWPSNR IWSSIM MAD MSE MSSIM NAE NQM SSIM SSIMmod VIF VIFP VSNR
t(ms) 2887.0 188.2 657.1 657.6 47640.0 4.5 132.7 4.3 373.6 25.7 25.1 1321.0 166.2 55.8

IQM2 release agreement

You can download the IQM2 here under the following terms:

  1. All documents and papers that report research results obtained using the IQM2 will acknowledge the use of the IQM2 measure. Please cite the following paper:
    E.Dumic, S.Grgic, M. Grgic, "IQM2: new image quality measure based on steerable pyramid wavelet transform and structural similarity index", Signal, Image and Video Processing, Vol. 8, No. 6, pp. 1159-1168, 2014
    The final publication is available at link.springer.com.
  2. While every effort has been made to ensure accuracy, IQM2 owners cannot accept responsibility for errors or omissions.
  3. Use of IQM2 measure is free of charge.
  4. IQM2 owners reserve the right to revise, amend, alter or delete the information provided herein at any time, but shall not be responsible for or liable in respect of any such revisions, amendments, alterations or deletions.

Repository

IQM2 measure (2 orientations, block size 5x5)

IQM2 measure (manually choose number of orientations and block size)