Autonomous Foraging with SARSA-based Deep Reinforcement Learning

XXII Symposium on Virtual and Augmented Reality (SVR)

Anderson Oliveira Mesquita¹   Yuri Lenon Barbosa Nogueira¹   Creto Augusto Vidal¹
Joaquim Bento Cavalcante Neto¹   Paulo Bruno de Sousa Serafim²

¹Department of Computing (DC), Federal University of Ceara (UFC)
²Instituto Atlântico

---

Page: [IEEE]

Abstract

This work proposes an autonomous system capable of foraging in an environment that has food and poisons distributed throughout a scenario. We use Deep Learning framework to process color images. These images simulate the agent's vision. The foraging task is modeled as a reinforcement learning problem, in which an input constituted by raw pixels is processed by a convolutional neural network resulting in an set of actions. An algorithm based on SARSA was used. During training, the agent selects the actions based on a probability distribution called Softmax. The objective of this work is to present an agent capable of searching for food and distinguishing it from poisons through continuous learning and without the help or external intervention from humans. The experiments show that the agent is able to distinguish food from poisons without the hints or markings in it's vision input. This highlights the advantages of combining Deep Learning with reinforcement learning for the foraging problem. The results of this work form an initial basis for understanding the relationship among autonomy, cognition and perception in artificial agents.

Video

Presentation starts at 1:47

BibTeX

Download

@InProceedings{mesquita2020autonomous,
  title = {Autonomous Foraging with {SARSA}-based Deep Reinforcement Learning},
  author  = {Mesquita, Anderson Oliveira and Nogueira, Yuri Lenon Barbosa and Vidal, Creto Augusto and Cavalcante-Neto, Joaquim Bento and Serafim, Paulo Bruno Sousa},
  booktitle = {Proceedings of the XXII Symposium on Virtual and Augmented Reality (SVR)},
  pages = {1--9},
  year = {2020},
  doi = {10.1109/SVR51698.2020.00070}
}