Cross-Modality Between Haptic and Auditory Roughness with a Force Feedback Device
Emi Kitamura*, Katsuya Miyashita**, Kenji Ozawa*,
Masaki Omata*, and Atsumi Imamiya*
*Interdisciplinary Graduate School of Medicine and Engineering, University of Yamanashi, 4-3-11 Takeda, Kofu, Yamanashi 400-8511, Japan
**Faculty of Engineering, University of Yamanashi, 4-3-11 Takeda, Kofu, Yamanashi 400-8511, Japan
Haptic roughness is basic to accurately identifying the texture of an object. When we manipulate everyday objects, their surfaces emit sound. Cross-modal effects between haptic and auditory roughness must thus be considered in realizing a multimodal human-computer interface. We conducted two experiments to accumulate basic data for the cross-modality using a force feedback device. In one experiment, we studied the cross-modal effect of auditory roughness on haptic roughness. We studied the effect of haptic roughness on auditory roughness in the other experiment. Results showed that cross-modal effects were mutually enhancing when their single-modal roughness was relatively high.
-  W. W. Gaver, “What in the world do we hear? An ecological approach to auditory event perception,” Ecol. Psycol., Vol.5, pp. 1-29, 1993.
-  W. W. Gaver, “How do we hear in the world? Explorations in ecological acoustics,” Ecol. Psycol., Vol.5, pp. 283-313, 1993.
-  V. Jousmäki, and R. Hari, “Parchment-skin illusion: sound-biased touch,” Curr. Biol., Vol.8, R190, 1998.
-  S. Guest, C. Carmur, D. Lloyd, and C. Spence, “Audiotactile interactions in roughness perception,” Exp. Brain Res., Vol.146, pp. 161-171, 2002.
-  S. J. Lederman, R. L. Klatzky, T. Morgan, and C. Hamilton, “Integrating multimodal information about surface texture via a probe: Relative contributions of haptic and touch-produced sound source,” Proc. HAPTICS’02, pp. 97-104, 2002.
-  M. R. McGee, P. Gray, and S. Brewster, “Mixed Feelings: Multimodal perception of virtual roughness,” Proc. Eurohaptics 2002, pp. 47-52, 2002.
-  S. J. Lederman, “Tactile roughness of grooved surfaces: The touching process and effects of macro- and microsurface structure,” Percept. Psychophys., Vol.16, pp. 385-395, 1974.
-  M. M. Taylor, and S. J. Lederman, “Tactile roughness of grooved surfaces: A model and the effect of friction,” Percept. Psychophys., Vol.17, pp. 23-36, 1975.
-  C. E. Connor, S. S. Hsiao, J. R. Phillips, and K. O. Johnson, “Tactile roughness: Neural codes that account for psychophysical magnitude estimates,” J. Neurosci., Vol.10, pp. 3823-3836, 1990.
-  T. Miyaoka, T. Mano, and M. Ohka, “Mechanisms of fine-surfacetexture discrimination in human tactile sensation,” J. Acoust. Soc. Am., Vol.105, pp. 2485-2492, 1999.
-  S. J. Lederman, R. L. Klatzky, C. L. Hamilton, and G. I. Ramsay, “Perceiving roughness via a rigid probe: Psychophysical effects of exploration speed and mode of touch,” Haptics-e (Electric Journal of Haptics Research), Vol.1, pp. 1-20, 1999.
-  M. R. McGee, P. Gray, and S. Brewster, “Feeling rough: Multimodal perception of virtual roughness,” Proc. Eurohaptics 2001, pp. 29-33, 2001.
-  E. Terhardt, “On the perception of periodic sound fluctuations (Roughness),” Acustica, Vol.30, pp. 201-213, 1974.
-  H. Fastl, “The hearing sensation roughness and neuronal responses to AM-tones,” Hearing Research, Vol.46, pp. 293-296, 1990.
-  E. Zwicker, and H. Fastl, “Psychoacoustics,” Springer-Verlag, pp. 231-236, 1990.
-  K. van den Doel, P. G. Kry, and D. K. Pai, “FoleyAutomatic: Physically-based sound effects for interactive simulation and animation,” Proc. SIGGRAPH 2001, pp. 537-544, 2001.
-  J. J. Zwislocki, and D. A. Goodman, “Absolute scaling of sensory magnitudes: A validation,” Perception & Psychophysics, Vol.28, pp. 8-38, 1980.
-  T. Hirahara, “Physical characteristics of headphones used in psychoacoustical experiments,” J. Acoustical Soc. Jpn., Vol.53, pp. 798-806, 1997 (in Japanese).
-  S. S. Stevens, “Issues in psychophysical measurement,” Psychological Review, Vol.78, pp. 426-450, 1971.
-  T. A. Ryan, “Interrelation of the sensory systems in perception,” Psychological Bulletin, Vol.37, pp. 659-698, 1940.
-  H. Fastl, “Fluctuation strength of modulated tones and broadband noise,” In R. Klinke, and R. Hartmann (Eds.), Hearing: Physiological Bases and Psychophysics, Berlin, Springer-Verlag, pp. 282-288, 1983.
-  M. Omata, M. Ishihara, M. G. Kwok, and A. Imamiya, “Haptizing wind on a weather map with reactive force and vibration,” Proc. IFIP INTERACT2005, pp. 19-29, 2005.
-  D. Morrill, “Trumpet algorithms for music composition,” Computer Music Journal, Vol.1, pp. 46-52, 1977.
-  S. J. Lederman, G. Thorne, and B. Jones, “Perception of texture by vision and touch: Multidimensionality and intersensory integration,” J. Experimental Psychology: Human Perception & Performance, Vol.12, pp. 169-180, 1986.
This article is published under a Creative Commons Attribution-NoDerivatives 4.0 Internationa License.
Copyright© 2006 by Fuji Technology Press Ltd. and Japan Society of Mechanical Engineers. All right reserved.