Investigating Deep Q-Network Agent Sensibility to Texture Changes on FPS Games

XIX Brazilian Symposium on Computer Games and Digital Entertainment (SBGames)

Paulo Bruno de Sousa Serafim1   Yuri Lenon Barbosa Nogueira2   Creto Augusto Vidal2
Joaquim Bento Cavalcante Neto2   Rômulo Freire Férrer Filho3

1Instituto Atlântico  2Department of Computing (DC), Federal University of Ceara (UFC)
3Teleinformatics Engineering Department (DETI), Federal University of Ceara (UFC)

Agents' sensibility

Paper: [PDF]  Page: [IEEE]

Abstract

Graphical updates are very common in modern digital games. For instance, PC game versions usually receive higher resolution textures after some time. This could be a problem for autonomous agents trained to play a game using Convolutional Neural Networks. These agents use the pixels of the screen as inputs and changing them could harm their performance. In this work, we evaluate agents' sensibility to texture changes. The agents are trained to play a First-Person Shooter game and then are presented to different versions of the same scenario, in which the only difference among them is texture changes. As the testbed, we use a ViZDoom scenario with a static monster that should be killed by the agent. Four agents are trained using Deep Q-Networks in four different scenarios. Then, every agent is tested in all four scenarios. We show that although every agent can learn the behaviors to win the game when playing the same version in which it was trained, they cannot generalize to all other versions. Only in one case, the agent had a good performance in a different scenario. Most of the time, the agent moved randomly or just stood still, and shot continuously, indicating that it could not understand the current screen. Even when the background textures were kept the same, the agent could not identify the enemy. Thus, to ensure proper behavior, an agent needs to be retrained not only if the problem changes, but also when only the visual aspects of the problem are modified.

Video

Presentation starts at 33:34

BibTeX

Download 

@InProceedings{serafim2020investigating,
  title = {Investigating Deep Q-Network Agent Sensibility to Texture Changes on {FPS} Games},
  author  = {Serafim, Paulo Bruno Sousa and Nogueira, Yuri Lenon Barbosa and Vidal, Creto Augusto and Cavalcante-Neto, Joaquim Bento and F\'{e}rrer Filho, R\^{o}mulo Freire},
  booktitle = {Proceedings of the XIX Brazilian Symposium on Computer Games and Digital Entertainment (SBGames)},
  pages = {1--9},
  year = {2020},
  issn = {2179-2259},
  doi = {10.1109/SBGames51465.2020.00025}
}

You may also enjoy