Automatic ROI Detection and Evaluation in Video Sequences Based on Human Interest

Mohammad Rokunuzzaman; Kosuke Sekiyama; Toshio Fukuda

doi:10.20965/jrm.2010.p0065

single-rb.php

« previous

JRM Vol.22 No.1 pp. 65-75

doi: 10.20965/jrm.2010.p0065

(2010)

Paper:

Views over last 60 days: 654

Automatic ROI Detection and Evaluation in Video Sequences Based on Human Interest

Mohammad Rokunuzzaman, Kosuke Sekiyama, and Toshio Fukuda

Dept. of Micro-Nano Systems Engineering, Nagoya University, Furo-cho, Nagoya-shi 464-8603, Japan

Received:

September 9, 2009

Accepted:

November 5, 2009

Published:

February 20, 2010

Keywords:

Region of Interest (ROI), attention, psychology of interest, psychology of relevance, ROI evaluation

Abstract

This paper describes an approach for automatic detection and evaluation of Region of Interest (ROI) based on human psychology of interest and relevance. It is very challenging to determine the cognitive boundary of the scene in real time due to its complexity in decision making. We have proposed a novel method to solve this problem by using human cognitive processes of attention, interest generation and relevancy. Our method successfully determines this cognitive boundary as ROI which is significant by its information content based on interest and relevancy. The effectiveness of ROI detection is checked with Eye tracker system. The highest detection accuracy found is 81.25% which is higher than the existing method. The ROI evaluation is compared with evaluation by human observers. The subjective correlation is found 85% with human evaluation. The experimental results show that our method is imitating human decision making for ROI detection and evaluation.

Cite this article as:

M. Rokunuzzaman, K. Sekiyama, and T. Fukuda, “Automatic ROI Detection and Evaluation in Video Sequences Based on Human Interest,” J. Robot. Mechatron., Vol.22 No.1, pp. 65-75, 2010.

Data files:

References

[1] S. Inagaki, T. Suzuki, T. Ito, and W. Shidan, “Design of Autonomous/Man-Machine-Cooperative Mobile Robot,” J. of Robotics and Mechatronics, Vol.21, No.2, pp. 252-259, 2009.
[2] D. J. Levitin, “Foundations of cognitive psychology: core readings, edition: illustrated,” MIT Press, 2002, ch.15, pp. 363-398.
[3] E. A. Styles, “The Psychology of Attention, edition: illustrated, reprint,” Psychology Press, 1997, ch.5, pp. 87-112.
[4] D. Sperber, F. Cara, and V. Girotto, “Relevance theory explains the selection task,” Cognition, 57, pp. 31-95, 1995.
[5] W. Osberger and A. J. Maeder, “Automatic identification of perceptually important regions in an image,” Proc. of the 11th Int. Conf. on Pattern Recognition, 1, pp. 701-704, 1998.
[6] L. Itti et al., “A model of saliency-based visual attention for rapid scene analysis,” IEEE Trans. on Pattern Analysis and Machine Intelligence, 20-11, pp. 1254-1259, 1998.
[7] M. Privitera and W. Stark, “Algorithms for Defining Visual Regions-of-Interest: Comparison with Eye Fixations,” IEEE Trans. on Pattern Analysis and Machine Intelligence, 22-9, pp. 970-982, 2000.
[8] H. Cheng et al., “Automatic video region-of-interest determination based on user attention model,” IEEE Int. Symposium on Circuits and Systems, 4, pp. 3219-3222, 2005.
[9] M. Privitera et al., “Locating regions-of-interest for the Mars Rover expedition,” Int. J. Remote sensing, 21-17, pp. 3327-3347, 2000.
[10] H. Liu et al., “Region-Based Visual Attention Analysis with Its Application in Image Browsing on Small Displays,” Proc. of the 15th Int. Conf. on Multimedia, pp. 305-308, 2007.
[11] T. Sevilmi et al., “Automatic detection of salient objects and spatial relations in videos for a video database system,” Int. J. of Image and Vision Computing, 26, pp. 1384-1396, 2008.
[12] N. Butko et al., “Visual Saliency Model for Robot Cameras,” Proc. of the Int. Conf. of Robotics and Automation, ICRA, pp. 2398-2403, 2008.
[13] Y. Li et al., “Salient Region Detection and Tracking in Video,” Int. Conf. on Multimedia and Expo, 2, pp. 269-272, 2003.
[14] M. Clauss, P. Bayerl, and H. Neumann, “A Statistical Measure for Evaluating Regions-of-Interest Based Attention Algorithms,” Pattern Recognition, 3175, pp. 383-390, 2004.
[15] R. H. Huesman, “A new fast algorithm for the evaluation of regions of interest and statistical uncertainty in computed tomography,” Phys. Med. Biol., 29-5, pp. 543-552, 1984.
[16] P. Correia and F. Pereira, “Estimation of Video Object’s Relevance,” Proc. of the European Signal processing conference (EUPSICO), Finland, pp. 925-928, 2000.
[17] Intel’s Open Computer Vision (OpenCV) Library,
Available: http://sourceforge.net/projects/opencvlibrary
[18] C. Stauffer and W. Grimson, “Adaptive Background Mixture Models for Real-Time Tracking,” Proc. of the IEEE Int. Conf. of Computer Vision and Pattern Recognition, pp. 246-252, 1999.
[19] P. Kaewtrakulpong and R. Bowden, “An Improved Adaptive Background Mixture Model for Real-time Tracking with Shadow Detection,” Proc. 2nd European Workshop on Advanced Video Based Surveillance Systems, AVBS01, pp. 135-144, 2001.
[20] S. T. Mueller and J. Zhang, “Upper and lower bounds of area under ROC curves and index of discriminability of classifier performance,” Proc. of the ICML 2006 Workshop on ROC Analysis in Machine Learning, pp. 41-46, 2006.

This article is published under a Creative Commons Attribution-NoDerivatives 4.0 Internationa License.

[1] [1] S. Inagaki, T. Suzuki, T. Ito, and W. Shidan, “Design of Autonomous/Man-Machine-Cooperative Mobile Robot,” J. of Robotics and Mechatronics, Vol.21, No.2, pp. 252-259, 2009.

[2] [2] D. J. Levitin, “Foundations of cognitive psychology: core readings, edition: illustrated,” MIT Press, 2002, ch.15, pp. 363-398.

[3] [3] E. A. Styles, “The Psychology of Attention, edition: illustrated, reprint,” Psychology Press, 1997, ch.5, pp. 87-112.

[4] [4] D. Sperber, F. Cara, and V. Girotto, “Relevance theory explains the selection task,” Cognition, 57, pp. 31-95, 1995.

[5] [5] W. Osberger and A. J. Maeder, “Automatic identification of perceptually important regions in an image,” Proc. of the 11th Int. Conf. on Pattern Recognition, 1, pp. 701-704, 1998.

[6] [6] L. Itti et al., “A model of saliency-based visual attention for rapid scene analysis,” IEEE Trans. on Pattern Analysis and Machine Intelligence, 20-11, pp. 1254-1259, 1998.

[7] [7] M. Privitera and W. Stark, “Algorithms for Defining Visual Regions-of-Interest: Comparison with Eye Fixations,” IEEE Trans. on Pattern Analysis and Machine Intelligence, 22-9, pp. 970-982, 2000.

[8] [8] H. Cheng et al., “Automatic video region-of-interest determination based on user attention model,” IEEE Int. Symposium on Circuits and Systems, 4, pp. 3219-3222, 2005.

[9] [9] M. Privitera et al., “Locating regions-of-interest for the Mars Rover expedition,” Int. J. Remote sensing, 21-17, pp. 3327-3347, 2000.

[10] [10] H. Liu et al., “Region-Based Visual Attention Analysis with Its Application in Image Browsing on Small Displays,” Proc. of the 15th Int. Conf. on Multimedia, pp. 305-308, 2007.

[11] [11] T. Sevilmi et al., “Automatic detection of salient objects and spatial relations in videos for a video database system,” Int. J. of Image and Vision Computing, 26, pp. 1384-1396, 2008.

[12] [12] N. Butko et al., “Visual Saliency Model for Robot Cameras,” Proc. of the Int. Conf. of Robotics and Automation, ICRA, pp. 2398-2403, 2008.

[13] [13] Y. Li et al., “Salient Region Detection and Tracking in Video,” Int. Conf. on Multimedia and Expo, 2, pp. 269-272, 2003.

[14] [14] M. Clauss, P. Bayerl, and H. Neumann, “A Statistical Measure for Evaluating Regions-of-Interest Based Attention Algorithms,” Pattern Recognition, 3175, pp. 383-390, 2004.

[15] [15] R. H. Huesman, “A new fast algorithm for the evaluation of regions of interest and statistical uncertainty in computed tomography,” Phys. Med. Biol., 29-5, pp. 543-552, 1984.

[16] [16] P. Correia and F. Pereira, “Estimation of Video Object’s Relevance,” Proc. of the European Signal processing conference (EUPSICO), Finland, pp. 925-928, 2000.

[17] [17] Intel’s Open Computer Vision (OpenCV) Library,
Available: http://sourceforge.net/projects/opencvlibrary

[18] [18] C. Stauffer and W. Grimson, “Adaptive Background Mixture Models for Real-Time Tracking,” Proc. of the IEEE Int. Conf. of Computer Vision and Pattern Recognition, pp. 246-252, 1999.

[19] [19] P. Kaewtrakulpong and R. Bowden, “An Improved Adaptive Background Mixture Model for Real-time Tracking with Shadow Detection,” Proc. 2nd European Workshop on Advanced Video Based Surveillance Systems, AVBS01, pp. 135-144, 2001.

[20] [20] S. T. Mueller and J. Zhang, “Upper and lower bounds of area under ROC curves and index of discriminability of classifier performance,” Proc. of the ICML 2006 Workshop on ROC Analysis in Machine Learning, pp. 41-46, 2006.