A Reinforcement Learning Scheme of Fuzzy Rules with Reduced Conditions

Abstract

This paper proposes a new method of Q-learning for the case where the states (conditions) and actions of systems are assumed to be continuous. The components of Q-tables are interpolated by fuzzy inference. The initial set of fuzzy rules is made of all combinations of conditions and actions relevant to the problem. Each rule is then associated with a value by which the Q-values of condition/action pairs are estimated. The values are revised by the Q-learning algorithm so as to make the fuzzy rule system effective. Although this framework may require a huge number of the initial fuzzy rules, we will show that considerable reduction of the number can be done by adopting what we call Condition Reduced Fuzzy Rules (CRFR). The antecedent part of CRFR consists of all actions and the selected conditions, and its consequent is set to be its Q-value. Finally, experimental results show that controllers with CRFRs perform equally well to the system with the most detailed fuzzy control rules, while the total number of parameters that have to be revised through the whole learning process is considerably reduced, and the number of the revised parameters at each step of learning increased.