dc.contributor.author | Kuo, Yen-Ling | |
dc.contributor.author | Barbu, Andrei | |
dc.contributor.author | Katz, Boris | |
dc.date.accessioned | 2022-03-24T17:07:22Z | |
dc.date.available | 2022-03-24T17:07:22Z | |
dc.date.issued | 2021-07-19 | |
dc.identifier.uri | https://hdl.handle.net/1721.1/141357 | |
dc.description.abstract | We demonstrate how a reinforcement learning agent can use compositional recurrent neural net- works to learn to carry out commands specified in linear temporal logic (LTL). Our approach takes as input an LTL formula, structures a deep network according to the parse of the formula, and determines satisfying actions. This compositional structure of the network enables zero-shot generalization to sig- nificantly more complex unseen formulas. We demonstrate this ability in multiple problem domains with both discrete and continuous state-action spaces. In a symbolic domain, the agent finds a sequence of letters that satisfy a specification. In a Minecraft-like environment, the agent finds a sequence of actions that conform to a formula. In the Fetch environment, the robot finds a sequence of arm config- urations that move blocks on a table to fulfill the commands. While most prior work can learn to execute one formula reliably, we develop a novel form of multi-task learning for RL agents that allows them to learn from a diverse set of tasks and generalize to a new set of diverse tasks without any additional training. The compositional structures presented here are not specific to LTL, thus opening the path to RL agents that perform zero-shot generalization in other compositional domains. | en_US |
dc.description.sponsorship | This material is based upon work supported by the Center for Brains, Minds and Machines (CBMM), funded by NSF STC award CCF-1231216. | en_US |
dc.publisher | Center for Brains, Minds and Machines (CBMM), Frontiers in Robotics and AI | en_US |
dc.relation.ispartofseries | CBMM Memo;127 | |
dc.title | Compositional RL Agents That Follow Language Commands in Temporal Logic | en_US |
dc.type | Article | en_US |
dc.type | Technical Report | en_US |
dc.type | Working Paper | en_US |