Assessing the Robustness of Deep Q-Network Agents to Changes on Game Object Textures

XX Brazilian Symposium on Computer Games and Digital Entertainment (SBGames)

Paulo Bruno de Sousa Serafim¹   Yuri Lenon Barbosa Nogueira²
Joaquim Bento Cavalcante Neto²   Creto Augusto Vidal²

¹Instituto Atlântico  ²Department of Computing (DC), Federal University of Ceara (UFC)

---

Paper: [PDF]  Page: [IEEE]

Abstract

The research in autonomous agents aspires to achieve Artificial General Intelligence, where agents, like humans, are able to understand concepts and learn how to solve tasks. We would like to observe this ability on game agents as well. Recent research on autonomous agents for game playing uses a combination of Deep Neural Networks and Reinforcement Learning algorithms. Commonly, Neural Networks present vision-based models, usually Convolutional Neural Networks (CNN). However, those models can undergo performance degradation when dealing with different pixel patterns, an issue that also happens with vision-based autonomous agents in games. Prior works have shown that CNN-based autonomous agents cannot reproduce the behavior learned in one scene when they are placed into a brand new version with different textures. In this work, we evaluate whether the agents educe high-level elements, such as enemy, foreground, and background. Instead of testing the agent in a completely different scene, we designed two experiments based on slight changes. In the first experiment, we change only a subset of the game objects. In the second experiment, the agents play in an interpolated version of two scenes. Even when changing only a single game object texture, the agents are not guaranteed to present good behavior. We show that, depending on the training scenario, the agents are not fully robust to generalize a high-level concept of game objects.

Video

Presentation starts at 49:04

BibTeX

Download

@InProceedings{serafim2021assessing,
  title = {Assessing the Robustness of Deep Q-Network Agents to Changes on Game Object Textures},
  author  = {Serafim, Paulo Bruno Sousa and Nogueira, Yuri Lenon Barbosa and Cavalcante-Neto, Joaquim Bento and Vidal, Creto Augusto},
  booktitle = {Proceedings of the XX Brazilian Symposium on Computer Games and Digital Entertainment (SBGames)},
  pages = {19--28},
  year = {2021},
  doi = {10.1109/SBGames54170.2021.00013}
}