An Attempt to Improve Food/Sound Congruity Using an Electromyogram Pseudo-Chewing Sound Presentation System
Hiroshi Endo, Hidekazu Kaneko, Shuichi Ino, and Waka Fujisaki
Human Informatics Research Institute, National Institute of Advanced Industrial Science and Technology (AIST)
Central 6, 1-1 Higashi, Tsukuba, Ibaraki 305-8566, Japan
Improving the texture of foods provided during nursing care is necessary to improve the appetite of elderly individuals. We developed a system to vary perceived food texture using pseudo-chewing sounds generated from electromyogram (EMG) signals. However, this previous system could not provide chewing sounds that were sufficiently congruous with foods. Because food/sound combinations that seem unnatural cause individuals to feel uncomfortable with pseudo-chewing sounds, food/sound congruity is important. This research aims to improve the derivation and presentation of pseudo-chewing sounds so as to be able to provide various kinds of chewing sounds. The developed system adjusts the volume of pseudo-chewing sounds that are stored in a digital audio player based on the amplitude of the EMG signal envelope. Using this system, food/sound congruity was examined with two kinds of softened Japanese pickles. Six kinds of pseudo-chewing sounds were tested (noisy chewing sound, EMG chewing sound, and four kinds of actual chewing sounds: rice cracker, cookie, and two kinds of Japanese pickles). Participants reported that food/sound combinations were unnatural with the noisy and EMG chewing sounds, whereas the combinations felt more natural with the pseudo-chewing sounds of Japanese pickles. We concluded that the newly developed system could effectively reduce the unnatural feeling of food/sound incongruity.
-  J. Curran and M. E. Groher, “Development and dissemination of an aspiration risk reduction diet,” Dysphagia, Vol.5, No.1, pp. 6-12, 1990.
-  J. M. Garcia and E. IV. Chambers, “Managing dysphagia through diet modifications,” Am. J. Nurs., Vol.110, No.11, pp. 26-33, 2010.
-  A. W. Martin, “Dietary management of swallowing disorders,” Dysphagia, Vol.6, No.3, pp. 129-134, 1991.
-  E. M. Pardoe, “Development of a multistage diet for dysphagia,” J. Am. Diet Assoc., Vol.93, No.5, pp. 568-571, 1993.
-  C. M. Steele, W. A. Alsanei, S. Ayanikalath, C. E. Barbon, J. Chen, et al., “The influence of food texture and liquid consistency modification on swallowing physiology and function: a systematic review,” Dysphagia, Vol.30, No.1, pp. 2-26, 2015.
-  J. Delwiche, “The impact of perceptual interactions on perceived flavor,” Food Qual. Prefer., Vol.15, No.2, pp. 137-146, 2004.
-  A. S. Szczesniak, “Texture is a sensory property,” Food Qual. Prefer., Vol.13, No.4, pp. 215-225, 2002.
-  N. Colodny, “Dysphagic independent feeders’ justifications for noncompliance with recommendations by a speech-language pathologist,” Am. J. Speech Lang. Pathol., Vol.14, No.1, pp. 61-70, 2005.
-  J. M. Garcia, E. IV. Chambers, and M. Molander, “Thickened liquids: practice patterns of speech-language pathologists,” Am. J. Speech Lang. Pathol., Vol.14, No.1, pp. 4-13, 2005.
-  L. Mioche, P. Bourdiol, and M. Peyron, “A. Influence of age on mastication: effects on eating behaviour,” Nutr. Res. Rev., Vol.17, No.1, pp. 43-54, 2004.
-  K. Swan, R. Speyer, B. J. Heijnen, B. Wagg, and R. Cordier, “Living with oropharyngeal dysphagia: effects of bolus modification on health-related quality of life–a systematic review,” Qual. Life Res., Vol.24, No.10, pp. 2447-2456, 2015.
-  T. Uemura, T. Moriya, H. Yano, and H. Iwata, “Development of a food simulator,” Trans. Virtual Reality Soc. Japan, Vol.8, No.4, pp. 399-406, 2003 (in Japanese).
-  Y. Hashimoto, J. Ohtaki, M. Kojima, N. Nagaya, T. Mitani, S. Miyajima, A. Yamamoto, and M. Inami, “Straw-like user interface: the display of drinking sensation,” Trans. Virtual Reality Soc. Japan, Vol.11, No.2, pp. 347-356, 2006 (in Japanese).
-  L. Duizer, “A review of acoustic research for studying the sensory perception of crisp, crunchy and crackly textures,” Trends in Food Science & Technology, Vol.12, No.1, pp. 17-24, 2001.
-  M. Zampini and C. Spence, “Assessing the role of sound in the perception of food and drink,” Chemosensory Perception, Vol.3, No.1, pp. 57-67, 2010.
-  M. Saeleaw and G. Schleining, “A review: crispness in dry foods and quality measurements based on acoustic-mechanical destructive techniques,” J. of Food Engineering, Vol.105, No.3, pp. 387-399, 2011.
-  C. Spence, “Eating with our ears: assessing the importance of the sounds of consumption on our perception and enjoyment of multisensory flavour experiences,” Flavor, Vol.4, No.3, pp. 1-14, 2015.
-  M. Zampini and C. Spence, “The role of auditory cues in modulating the perceived crispness and staleness of potato chips,” J. of Sensory Studies, Vol.19, No.5, pp. 347-363, 2004.
-  M. Masuda and K. Okajima, “Effects of mastication sound in food texture perception and pleasantness,” IEICE Tech. Report, Vol.111, No.285, pp. 57-62, 2011 (in Japanese).
-  N. Koizumi, H. Tanaka, Y. Uema, and M. Inami, “Chewing JOCKEY - designing user interface to augment food texture with sound AR system -,” Trans. Virtual Reality Soc. Japan, Vol.18, No.2, pp. 141-150, 2013 (in Japanese).
-  M. L. Dematte, N. Pojer, I. Endrizzi, M. L. Corollaro, E. Betta, et al., “Effects of the sound of the bite on apple perceived crispness and hardness,” Food Quality and Preference, Vol.38, pp. 58-64, 2014.
-  S. Deneve and A. Pouget, “Bayesian multisensory integration and cross-modal spatial links,” J. of Physiology, Vol.98, No.1-3, pp. 249-258, 2004.
-  M. O. Ernst and H. H. Bulthoff, “Merging the senses into a robust percept,” Trends in Cognitive Sciences, Vol.8, No.4, pp. 162-169, 2004.
-  H. Endo, S. Ino, and W. Fujisaki, “The effect of a crunchy pseudo-chewing sound on perceived texture of softened foods,” Physiology & Behavior, Vol.164, pp. 324-331, 2016.
-  H. Endo, S. Ino, and W. Fujisaki, “Improving the palatability of nursing care food using a pseudo-chewing sound generated by an EMG signal,” V. G. Duffy (Ed.), Digital Human Modeling: Applications in Health, Safety, Ergonomics and Risk Management, Proc. HCI Int. 2016, Springer, pp. 212-220, 2016.