Machine learning has been attracting significant attention again since the potential of deep learning was recognized. Not only has machine learning been improved, but it has also been integrated with “reinforcement learning,” revealing other potential applications, e.g., deep Q-networks (DQN) and AlphaGO proposed by Google DeepMind. It is against this background that this special issue, “Cutting Edge of Reinforcement Learning and its Hybrid Methods,” focuses on both reinforcement learning and its hybrid methods, including reinforcement learning with deep learning or evolutionary computation, to explore new potentials of reinforcement learning.

Of the many contributions received, we finally selected 13 works for publication. The first three propose hybrids of deep learning and reinforcement learning for single agent environments, which include the latest research results in the areas of convolutional neural networks and DQN. The fourth through seventh works are related to the Learning Classifier System, which integrates evolutionary computation and reinforcement learning to develop the rule discovery mechanism. The eighth and ninth works address problems related to goal design or the reward, an issue that is particularly important to the application of reinforcement learning. The last four contributions deal with multiagent environments.

These works cover a wide range of studies, from the expansion of techniques incorporating simultaneous learning to applications in multiagent environments. All works are on the cutting edge of reinforcement learning and its hybrid methods. We hope that this special issue constitutes a large contribution to the development of the reinforcement learning field.

Six years have passed since the publication of our Special Issue on Human-Robot Interaction Systems in 2011. Since then, artificial intelligence and robotics have developed rapidly, and the opportunities for human beings and robots to work together have increased. The objective of this special issue’s twelve articles is to activate and expand high-quality research.

In the first article, Y. Tamura, T. Akashi, and H. Osumi propose a computational model of robot’s gaze. In the second article, S. Hoshino and K. Uchida propose an interactive motion planner for robot navigation in dynamic environments. In the third article, T. Iio, Y. Yoshikawa, and H. Ishiguro develop a conversational robotic system based on human response. In the fourth article, K. Sakai, F. Dalla Libera, Y. Yoshikawa, and H. Ishiguro propose a method for generating bystander robots’ actions that is based on an analysis of the relative probabilities of human responses to robot actions. In the fifth article, T. Matsumaru and M. Narita present a newly developed support system for learning calligraphy strokes. In the sixth article, E. Tamura, Y. Yamashita, T. Yamashita, E. Sato-Shimokawara, and T. Yamaguchi present a method of driving a car simply by gesturing. In the seventh article, A. Kurosu and T. Hashimoto develop an eye robot with two degrees of freedom. It is intended for use as a communication robot. In the eighth article, T. Hashimoto, Y. Munakata, R. Yamanaka, and A. Kurosu report on a method for retrieving episodic memories. In the ninth article, Y. Nishikawa, Y. Kagawa, and A. Okazaki develop a spiral movement robot for inpatients. In the tenth article, Y. Umesawa, K. Doi, and H. Fujimoto develop an interface device that creates kinaesthetic illusions by inducing vibrations in muscle tendons, vibrations that coordinate with dual-joint movements. In the eleventh article, R. Horio, N. Uchiyama, and S. Sano propose a human-operated biped robot for transporting objects over rough terrain or up steps. In the closing contribution, T. Sakuraba, N. Uchiyama, S. Sano, and T. Sakaguchi present the design of a spring-based regenerativ...<more>

The JACIII was first published in 1997, and 2017 marks its 20^th anniversary. During the last two decades, the research fields in computational intelligence have rapidly evolved owing to the spread of the Internet, performance improvement of computers, and accumulation of scientific knowledge. To celebrate this 20^th anniversary, we have selected 6 important research areas from the JACIII scope, and invited outstanding researchers from each of these areas to contribute papers about the progress and major topics in those areas during the past 20 years. Submitted paper went through a peer-review process by distinguished professors to further improve the quality. The research areas selected were computational intelligence, fuzzy intelligence, intelligent robots, artificial intelligence and web intelligence, data mining, and smart grids. Each of those paper covers broad topics appeared in the research areas, from which readers could grasp what happened during the past 20 years. We also hope readers could find some hints about future directions of their own researches towards the next 20 years.

Invited Paper 1: Computational Intelligence: Retrospection and Future

Author: Witold Pedrycz (University of Alberta, Canada)

Invited Paper 2: Fuzzy Inference: Its Past and Prospects

Authors: Kiyohiko Uehara (Ibaraki University, Japan) and Kaoru Hirota (Beijing Institute of Technology, China)

Invited Paper 3: Relationship Between Human and Robot in Nonverbal Communication

Authors: Yukiko Nakagawa and Noriaki Nakagawa (RT Corporation, Japan)

Invited Paper 4: Web Intelligence and Artificial Intelligence

Author: Yasufumi Takama (Tokyo Metropolitan University, Japan)

Invited Paper 5: A Review of Data Mining Techniques and Applications

Authors: Ratchakoon Pruengkarn, Kok Wai Wong, and Chun Che Fung (Murdoch University, Australia)

Invited Paper 6: Development and Current State of Smart Grids: A Review

Author: Ken Nagasaka (Tokyo University of Agriculture and Technology, Japan)