Minh Anh T. Ho, Yoji Yamada, and Yoji Umetani

Abstract:
In our original system, we used hidden Markov models (HMMs) to model rough gesture patterns. We later utilized temporal difference (TD) learning to adjust the action model of the tracker for its behavior in the tracking task. We integrated the above two methods into an algorithm by assigning state transition probability in HMMs as a reward in TD learning. Identification of the sign gesture context through wavelet analysis autonomously provides a reward value for optimizing the attentive visual attentive tracker's AVAT's action patterns. A bound of state value functions as a constraint factor for the updating procedure in TD models has been determined to recognize whether predictive models need to be updated according with action models. Experimental results of extracting an operator's hand sign sequence during natural walking demonstrates AVAT development in the perceptual organization framework.

Keywords: visual tracking, intended gestures, hidden Markov model, reinforcement learning, state transition update