Rex (@RexGLiu) header
Rex (@RexGLiu) profile picture
Joined June 2020
We also apply our approach to a more complex "castles" task with hierarchical goal-subgoal structure.
1
Here, we look at how humans can compositionally generalise over models of RL tasks (in this case, compositions of MDP reward and transition functions).
1
Across multiple simulations, our hierarchical agent (purple) is able to do better than an agent that only learns about components but not structures (red) and an agent that doesn't transfer at all (green).
1
2
The agent starts in the upper level and has to visit each sublevel in the right order. It navigates to the first door, enters a sublevel, travels to the sublevel exit (orange star), returns to the upper level, goes to the second door, etc.
1
2
We tried our approach on a simple gridworld task first, where agents had to combine different transition functions and with different goals.
1
2
But at the lower levels, it also lets us combine different components together within different structures (condition 2).
1
1
And we can leverage this compositionality to solve new tasks. By taking fingerings from the flute and songs from the guitar, we can make a reasonable first attempt at the piccolo.
1
1
Most tasks we encounter in daily life have a compositional structure. Imagine playing a musical instrument. You can break this down into two parts: what songs you want to play and what fingerings to use.
1
1