Hierarchical Reinforcement Learning: Learning Sub-goals and State-Abstraction

Abstract

In this paper we present a method that allows an agent to discover and create temporal abstractions autonomously. Our method is based on the concept that to reach the goal, the agent must pass through relevant states that we will interpret as subgoals. To detect useful subgoals, our method creates intersections between several paths leading to a goal. Our research focused on domains largely used in the study of temporal abstractions. We used several versions of the room-to-room navigation problem. We determined that, in the problems tested, an agent can learn more rapidly by automatically discovering subgoals and creating abstractions.