HMM-based Temporal Difference Learning with State Transition Updating for Tracking Human Communicational Behaviors
Minh Anh T. Ho, Yoji Yamada, and Yoji Umetani
Intelligent Systems Laboratory, Graduate School of Toyota Technological Institute, 2-12 Hisakata, Tenpa-ku, Nagoya, 468-2511 Japan
In our original system, we used hidden Markov models (HMMs) to model rough gesture patterns. We later utilized temporal difference (TD) learning to adjust the action model of the tracker for its behavior in the tracking task. We integrated the above two methods into an algorithm by assigning state transition probability in HMMs as a reward in TD learning. Identification of the sign gesture context through wavelet analysis autonomously provides a reward value for optimizing the attentive visual attentive tracker’s AVAT’s action patterns. A bound of state value functions as a constraint factor for the updating procedure in TD models has been determined to recognize whether predictive models need to be updated according with action models. Experimental results of extracting an operator’s hand sign sequence during natural walking demonstrates AVAT development in the perceptual organization framework.
This article is published under a Creative Commons Attribution-NoDerivatives 4.0 International License.
Copyright© 2003 by Fuji Technology Press Ltd. and Japan Society of Mechanical Engineers. All right reserved.