Advanced Reinforcement Learning

The goal of this proposal is to improve the theory and practice of Reinforcement Learning (RL). Our extraordinarily successful SNF RL projects so far have led to 60 publications since 2009, four of which won awards. RL is closely related to how animals and humans act and learn. Without a teacher, solely from occasional real-valued pain and pleasure signals, RL agents must discover how to interact with a dynamic environment to maximize their future expected reward. The classical approach to RL makes unrealistic strong assumptions such as: the current input of the agent tells it all needs to know about the environment. Our more general methods learn to create memories of important events, solving numerous RL/optimization tasks unsolvable by classical RL methods. The originally proposed project subgoals included: (1) Improve scalability of state-of-the-art RL controllers towards high-dimensional observation/action spaces, using Compressed Network Search and unsupervised redundancy reduction through slow feature analysis; (2) Learn through RL to actively focus an internal searchlight of attention, to improve state-of-the-art vision systems; (3) Make further steps towards implementing a full-fledged, self-improving Goedel Machine; (4) Implement and test novel Optimal Ordered Problem Solver-like systems with a programming language based on self-delimiting nets that determine their own effective size and runtime during search for solutions to optimisation problems.