DRLeague: a Novel 3D Environment for Training Reinforcement Learning Agents

XXI Brazilian Symposium on Computer Games and Digital Entertainment (SBGames)

Hyuan Peixoto Farrapo1,2     Rômulo Freire Férrer Filho1,3
José Gilvan Rodrigues Maia1,2   Paulo Bruno de Sousa Serafim4

1Federal University of Ceará (UFC)  2Virtual UFC Institute
3Department of Computing (DC)  4COATI, Inria centre at Université Côte d'Azur

DRLeague


Abstract

The development of autonomous agents performing unique interactions that resemble human-like behavior is currently driven by Deep Reinforcement Learning (DRL) techniques combined with complex virtual environments. These constitute an active field of research that is fueled by environments usually inspired or borrowed from video games. Although works in the area commonly do not make use of trending 3D games, these games are interesting testbeds for more complex and compelling behaviors, as they tend to explore more variables than their predecessors. This paper introduces DRLeague, a novel DRL environment, proposed to be open-source, and easily customizable, which supports mechanics for 3D games inspired by the popular “car football” game Rocket League. Besides the typical gameplay, we implemented four challenging minigames based on the mechanics from this title with advanced physics simulation and fine-grained car control: penalty shoot, multiplayer penalty shoot, barrier kick, and aerial shoot, each of these requiring more complex skills than the previous ones. Finally, we provide solid baseline experimental results showing the learning progress of agents using Unity’s ML-Agents toolkit, evidencing DRLeague as a suitable testbed in the application of machine learning techniques.

BibTeX

Download

@InProceedings{farrapo2022drleague,
  title = {DRLeague: a Novel 3D Environment for Training Reinforcement Learning Agents},
  author  = {Farrapo, Hyuan Peixoto and F\'{e}rrer Filho, R\^{o}mulo Freire and Maia, Jos\'{e} Gilvan Rodrigues and Serafim, Paulo Bruno Sousa},
  booktitle = {Proceedings of the XXI Brazilian Symposium on Computer Games and Digital Entertainment (SBGames)},
  pages = {1--6},
  year = {2022}
}