Projects

Current Projects

WorldDynamics.jl

WorldDynamics.jl is an open-source framework for world dynamics modeling and simulation. This project aims to provide a modern framework to investigate integrated assessment models of sustainable development, based on current software engineering and scientific machine learning techniques. Led by Pierluigi Crescenzi, Emanuele Natale, and I, the group is developing a Julia library to allow scientists to use and adapt different world models easily. By enabling an open, interdisciplinary, and consistent comparative approach to scientific model development, our goal is to inform global policy makers on environmental and economic issues.

Repository: WorldDynamics.jl GitHub

Documentation: WorldDynamics.jl Docs

Interpretability of DRL agents

One of my research interests is applying Explainable Artificial Intelligence (XAI) methods to autonomous agents. In particular, for agents developed using Deep Reinforcement Learning (DRL). This research branch started with a work in 2020 by investigating how the agents would perform in the same scenario but with different textures1. Then, a more rigorous assessment of how specific game object appearance affects an agent's performance2 followed. At this moment, XAI applied to DRL is the subject of Alexandre Magno Master's works.

Related publications

  1. Assessing the Robustness of Deep Q-Network Agents to Changes on Game Object Textures
  2. Investigating Deep Q-Network Agent Sensibility to Texture Changes on FPS Games

Past Projects

DRLeague

DRLeague is a novel DRL environment, proposed to be open-source, and easily customizable, which supports mechanics for 3D games inspired by the popular “car football” game Rocket League. Besides the typical gameplay, there are four minigames based on the game mechanics with advanced physics simulation and fine-grained car control: penalty shoot, multiplayer penalty shoot, barrier kick, and aerial shoot. DRLeague started as a work for the Bachelor's thesis of Hyuan Farrapo. Then, the work was extended to the current environment version.

Repository: DRLeague GitHub

Publication: DRLeague: a Novel 3D Environment for Training Reinforcement Learning Agents

GymHero

Gym Hero is a Reinforcement Learning environment based on the game Guitar Hero. It consists of a similar game implementation, developed using PyGame, with four difficulty levels, and able to randomly generate tracks. On top of the game, a Gym environment was implemented to allow training and evaluation of Reinforcement Learning agents. GymHero was primarily the effort of Rômulo Férrer Filho in his Bachelor's thesis.

Repository: GymHero GitHub

Publication: Gym Hero: A Research Environment for Reinforcement Learning Agents in Rhythm Games