Objective Image Quality Assessment Based on Saliency Map
Longsheng Wei*,**, Wei Liu*,**, Xinmei Wang*, Feng Liu*, and Dapeng Luo***
*School of Automation, China University of Geosciences
Wuhan 430074, China
**Computational Intelligence & Information Processing Laboratory, China University of Geosciences
Wuhan 430074, China
***Faculty of Mechanical and Electronic Information, China University of Geosciences
Wuhan 430074, China
The development of objective image quality assessment metrics aligned with human perception is of fundamental importance to numerous image processing applications. In this paper, an objective image quality assessment approach based on saliency map is proposed. By local shift estimation method, the retargeted image is resized to the same size as the reference image. A gradient magnitude similarity map is computed by comparing the retargeted and reference images. The more similarly, the brighter of pixels in the gradient magnitude similarity map. At the same time, a saliency map of reference image is achieved by visual attention. Finally, an overall image quality score is computed from the gradient magnitude similarity map via saliency pooling strategy. The most important step in our approach is to generate a gradient magnitude similarity map that indicates at each spatial location in the source image how the structural information is preserved in the retargeted image. There are two key contributions in this paper, one is that we add the texture feature in computing saliency map because image gradient is very sensitive to texture information, and the other is that we propose a new objective image quality metrics by introducing saliency map into image quality evaluation. Experimental results indicate that the evaluation indexes of our approach are better than existing methods in the literature.
-  P. Zhang, W. Zhou, L. Wu, and H. Li, “SOM: Semantic Obviousness Metric for Image Quality Assessment,” IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 2394-2402, 2015.
-  A. Shamir, and O. Sorkine, “Visual media retargeting,” ACM SIGGRAPH Asia Courses, 2009.
-  Y. Fang, K. Zeng, Z. Wang, W. Lin, Z. Fang, and C. Lin, “Objective Quality Assessment for Image Retargeting Based on Structural Similarity,” IEEE J. on Emerging and Selected Topics in Circuits and Systems, Vol.4, No.1, pp. 95-104, 2014.
-  L. Zhang, D. Zhang, and X. Mou, “Fsim: a feature similarity index for image quality assessment,” IEEE Trans. on Image Processing, Vol.20, No.8, pp. 2378-2386, 2011.
-  S. Avidan, and A. Shamir, “Seam carving for content-aware image resizing,” ACM Trans. Graph., Vol.26, No.3, p.10, 2007.
-  Y. Pritch, E. Kav-Venaki, and S. Peleg, “Shift-map image editing,” Proc. IEEE 12th Int. Conf. on Computer Vision, pp. 151-158, 2009.
-  C. Barnes, E. Shechtman, A. Finkelstein, and D. Goldman, “Patch-Match: A randomized correspondence algorithm for structural image editing,” ACM Trans. on Graphics, Vol.28, No.3, p. 24, 2009.
-  Y. Fang, Z. Chen, W. Lin, and C. Lin, “Saliency detection in the compressed domain for adaptive image retargeting,” IEEE Trans. on Image Processing, Vol.21, No.9, pp. 3888-3901, September 2012.
-  M. Rubinstein, A. Shamir, and S. Avidan, “Multi-operator media retargeting,” ACM Trans. on Graphics, Vol.28, No.3, pp. 1-11, 2009.
-  W. Dong, N. Zhou, J. C. Paul, and X. Zhang, “Optimized image resizing using seam carving and scaling,” ACM Trans. on Graphics, Vol.28, No.5, pp. 1-10, 2009.
-  Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: From error visibility to structural similarity,” IEEE Trans. on Image Processing, Vol.13, No.4, pp. 600-612, 2004.
-  H. R. Sheikh, and A. C. Bovik, “Image information and visual quality,” IEEE Trans. on Image Processing, Vol.15, No.2, pp. 430-444, 2006.
-  W. Xue, L. Zhang, X. Mou, and A. Bovik, “Gradient magnitude similarity deviation: A highly efficient perceptual image quality index,” IEEE Trans. on Image Processing, Vol.23, No.2, pp. 684-695, 2014.
-  D. G. Lowe, “Object recognition from local scale-invariant features,” Int. Conf. on Computer Vision, Vol.2, pp. 1150-1157, 1999.
-  C. Liu, J. Yuen, and A. Torralba, “SIFT flow: Dense correspondence across different scenes and its applications,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol.33, No.5, 2011.
-  S. A. Coleman, B. W. Scotney, and S. Suganthan, “Multi-scale edge detection on range and intensity images,” Pattern Recognition, Vol.44, No.4, pp. 821-838, 2011.
-  L. Itti, C. Koch, and E. Niebur, “A model of saliency-based visual attention for rapid scene analysis,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol.20, No.11, pp. 1254-1259, 1998.
-  T. M”aenp”a”a, M.Pietik”ainen, “Texture analysis with local binary patterns,” C. Chen and P. Wang (Eds.), Handbook of Pattern Recognition and Computer Vision, World Scientific, 3rd edition, pp. 197-216, 2005.
-  L. Wei and D. Luo, “A biologically inspired computational approach to model top-down and bottom-up visual attention,” Optik – Int. J. for Light and Electron Optics, Vol.126, No.5, pp. 522-529, 2015.
-  Z. Wang and Q. Li, “Information content weighting for perceptual image quality assessment,” IEEE Trans. on Image Processing, Vol.20, No.5, pp. 1185-1198, 2011.
-  J. Park, K. Seshadrinathan, S. Lee, and A. C. Bovik, “VQpooling: Video quality pooling adaptive to perceptual distortion severity,” IEEE Trans. on Image Processing, Vol.22, No.2, pp. 610-620, 2013.
-  S. Castillo, T. Judd, and D.Gutierrez, “Using eye-tracking to assess different image retargeting methods,” ACM SIGGRAPH Symp. Appl. Percept. Graph. Visualizat., Toulouse, France, pp. 7-14, 2011.
-  M. Rubinstein, D. Gutierrez, O. Sorkine, and A. Shamir, “A comparative study of image retargeting,” Proc. ACM SIGGRAPH Asia, Vol.29, No.6, pp. 1-9, 2010.
-  M. G. Kendall, “A new measure of rank correlation,” Biometrika, pp. 81-93, 1938.