Tool Operation Recognition Based on Robust Optical Flow and HMM from Short-Time Sequential Image Data

Abstract

This paper discusses a method to precisely recognize which tool is to be used based on the optical flow and HMM from short-time sequential images that operate a variety of hand-operated carpenter tools in the real environment. Operation recognition from a single-eye camera includes problems on differences in the difficulty of fixing the shape of the tool and poor motion periodicity due to occlusion of the fingers, back of the hand, and arm of the operator. This paper models operation without separating the integrated motions of the hand and tool and recognizes it with four tools divided into different categories from these motions. The optical flow method via the nonlinear robust function is used to suppress possible error caused by discontinuous motion components, HMM with a flexible time axis is applied to implement learning and recognition. The average vector of the optical flow mapped into the conversion diagram was designed to output symbol numbers for the generation of symbol time series. The three subjects have been asked to operate given tools for conducting learning recognition experiments. The number of input and output symbol has been varied for comparison. This results in a maximum of 100% recognition on the average for learning and recognition by the same person and in a maximum of 88.6% for learning and recognition by different persons. This method has been proven robust and effective because an average of 79.4% or higher recognition rate has been obtained even for short data of input symbol 5 (equivalent to 0.2 seconds) difficult to recognize.