Q-Cog: a Q-Learning based cognitive agent architecture for complex 3D virtual worlds.

Abstract

Intelligent cognitive agents should be able to autonomously gather new knowledge and learn from their own experiences in order to adapt to a changing environment. 3D virtual worlds provide complex environments in which autonomous software agents may learn and interact. In many applications within this domain, such as video games and virtual reality, the environment is partially observable and agents must make decisions and react in real-time. Due to the dynamic nature of virtual worlds, adaptability is of great importance for virtual agents. The Reinforcement Learning paradigm provides a mechanism for unsupervised learning that allows agents to learn from their own experiences in the environment. In particular, the Q-Learning algorithm allows agents to develop an optimal action-selection policy based on their experiences. This research explores the adaptability of cognitive architectures using Reinforcement Learning to construct and maintain a library of action-selection policies. The proposed cognitive architecture, Q-Cog, utilizes a policy selection mechanism to develop adaptable 3D virtual agents. Results from experimentation indicates that Q-Cog provides an effective basis for developing adaptive self-learning agents for 3D virtual worlds.