About me

I am a PhD candidate at TU Delft with Frans Oliehoek and Matthijs Spaan.

My research focuses on (Multi-Agent) Reinforcement Learning with a focus on factored representations. I investigate how to abstract (factorize) the agent(s)’s state space to enable more effective learning and optimize runtime. I am also interested in causality, generalization, partial observability, and memory.

Last year I was an intern at JP Morgan AI Research in London. In 2021, I interned at Huawei Ireland Research Center. Before joining my PhD program, I worked as a data scientist at Unity in Copenhagen.

Research

Bad Habits: Policy Confounding and Out-of-Trajectory Generalization in RL.

M. Suau, M. T. J Spaan, F. A. Oliehoek. Preprint.

Distributed Influence-Augmented Local Simulators for Parallel MARL in Large Networked Systems.

M. Suau, J. He, M. M. Çelikok, M. T. J Spaan, F. A. Oliehoek. NeurIPS 2022.

Influence-Augmented Local Simulators: A Scalable Solution for Fast Deep RL in Large Networked Systems.

M. Suau, J. He, M. T. J Spaan, F. A. Oliehoek. ICML 2022.

Online Planning in POMDPs with Self-Improving Simulators.

J. He, M. Suau, H. Baier, M. Kaisers, F. A. Oliehoek. IJCAI 2022.

Speeding up Deep RL through Influence-augmented Local Simulators.

M. Suau, J. He, M. T. J. Spaan, F. A. Oliehoek. AAMAS 2022.

Influence-aware Memory Architectures for Deep Reinforcement Learning in POMDPs.

M. Suau, E. Congeduti, J. He, R.A.N. Starre, A. Czechowski, F. A. Oliehoek. NCAA 2022.

Offline Contextual Bandits for Wireless Network Optimization

M. Suau, A. Agapitos, D. Lynch, D. Farrell, M. Zhou, A. Milenovic. Offline RL workshop, NeurIPS 2021.

Influence-augmented Online Planning for Complex Environments.

J. He, M. Suau, F.A. Oliehoek. NeurIPS 2020.

Selected talks

• Berkeley MARL Seminar 2022 (online) - Scaling up MARL: Distributed Simulation of Large Networked Systems.
• ICML 2022 (Baltimore) - Influence-Augmented Local Simulators.
• CIG 2018 (Maastricht) - Unity ML Agents Tutorial.

Service

Teaching:

• From 2020 to 2022 I gave a series of lectures on Deep Learning at the Computational Intelligence course (CSE2530) at TU Delft.

Supervising:

• I Supervised three Bachelor’s and two Master’s thesis:

  • Nele Albers, Deniz Hofmeister, Sven Holtrop, Lucas Crijns, Cian Jansen.

Reviewing:

• ICML 2021, 2022 (received reviewing award).
• NeurIPS 2021, 2022 (received reviewing award).
• ICLR 2021, 2022.
• AAMAS 2022.