Sotabase

Career

· Senior Staff Research Scientist, Meta2025–
· Research Scientist, Google DeepMind2022–
· Research Intern, DeepMind2021–2021
· Research Scientist, Element AI2020–2020
· Visiting Research Scientist, Element AI2020–2020
· Visiting Researcher, Max Planck Institutes Tübingen2019–2019
· Visiting Researcher, University of California, Berkeley2018–2018
· Graduate Student Researcher, Mila - Quebec Artificial Intelligence Institute2016–2022
· Doctor of Philosophy - PhD, Deep Learning, Université de Montréal2016–2021
· Software Engineering Intern (Research), Google2014–2014
· Bachelor of Technology (B.Tech. Hons ), Computer Science, International Instit

Publications (150)

The Llama 3 Herd of Models
2024
12,282
cited
Toward Causal Representation Learning
Proceedings of the IEEE · 2021
1,228
cited
An Actor-Critic Algorithm for Sequence Prediction
International Conference on Learning Representations · 2016
661
cited
Professor Forcing: A New Algorithm for Training Recurrent Networks
Neural Information Processing Systems · 2016
649
cited
Inductive biases for deep learning of higher-level cognition
Proceedings of the Royal Society A · 2020
421
cited
A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms
International Conference on Learning Representations · 2019
364
cited
Recurrent Independent Mechanisms
International Conference on Learning Representations · 2019
353
cited
Towards Causal Representation Learning
arXiv.org · 2021
344
cited
Zoneout: Regularizing RNNs by Randomly Preserving Hidden Activations
International Conference on Learning Representations · 2016
324
cited
Monte Carlo Tree Search Boosts Reasoning via Iterative Preference Learning
arXiv.org · 2024
203
cited
Z-Forcing: Training Stochastic Recurrent Networks
Neural Information Processing Systems · 2017
193
cited
Learning Neural Causal Models from Unknown Interventions
arXiv.org · 2019
184
cited
InfoBot: Transfer and Exploration via the Information Bottleneck
International Conference on Learning Representations · 2019
176
cited
CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and Transfer Learning
International Conference on Learning Representations · 2020
136
cited
On Disentangled Representations Learned from Correlated Data
International Conference on Machine Learning · 2020
129
cited
Maximum Entropy Generators for Energy-Based Models
arXiv.org · 2019
117
cited
A Theory for Emergence of Complex Skills in Language Models
arXiv.org · 2023
105
cited
Sparse Attentive Backtracking: Temporal CreditAssignment Through Reminding
Neural Information Processing Systems · 2018
92
cited
Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templates
Neural Information Processing Systems · 2024
91
cited
Small-GAN: Speeding Up GAN Training Using Core-sets
International Conference on Machine Learning · 2019
88
cited
Sotabase