Sotabase

Career

· Core Academic Member / Canada CIFAR AI Chair, Mila - Quebec Artificial Intelligence Institute2020–
· Assistant Professor, McGill University2019–
· PhD Student, University of Edinburgh2012–2017
· Google PhD Fellow, Google
· Research Scientist, ServiceNow Research
· Software Engineer, Sketch Engine
· Postdoctoral Researcher, Stanford University

Publications (123)

Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
arXiv.org · 2022
2,201
cited
CoQA: A Conversational Question Answering Challenge
Transactions of the Association for Computational Linguistics · 2018
1,330
cited
StarCoder: may the source be with you!
Trans. Mach. Learn. Res. · 2023
1,079
cited
LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders
arXiv.org · 2024
421
cited
Universal Dependencies 2.1
2017
418
cited
CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies
Conference on Computational Natural Language Learning · 2017
341
cited
The Impact of Positional Encoding on Length Generalization in Transformers
Neural Information Processing Systems · 2023
315
cited
End-to-End Training of Multi-Document Reader and Retriever for Open-Domain Question Answering
Neural Information Processing Systems · 2021
185
cited
Evaluating Correctness and Faithfulness of Instruction-Following Models for Question Answering
Transactions of the Association for Computational Linguistics · 2023
179
cited
An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Models
Annual Meeting of the Association for Computational Linguistics · 2021
147
cited
Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies
CoNLL Shared Task · 2017
137
cited
TopiOCQA: Open-domain Conversational Question Answering with Topic Switching
Transactions of the Association for Computational Linguistics · 2021
119
cited
Understanding by Understanding Not: Modeling Negation in Language Models
North American Chapter of the Association for Computational Linguistics · 2021
103
cited
MMTEB: Massive Multilingual Text Embedding Benchmark
arXiv.org · 2025
77
cited
Universal Dependencies 2.0 – CoNLL 2017 Shared Task Development and Test Data
2017
68
cited
Measuring Systematic Generalization in Neural Proof Generation with Transformers
Neural Information Processing Systems · 2020
67
cited
DeepSeek-R1 Thoughtology: Let's think about LLM Reasoning
2025
57
cited
VinePPO: Refining Credit Assignment in RL Training of LLMs
International Conference on Machine Learning · 2024
52
cited
Using In-Context Learning to Improve Dialogue Safety
Conference on Empirical Methods in Natural Language Processing · 2023
50
cited
Evaluating the Faithfulness of Importance Measures in NLP by Recursively Masking Allegedly Important Tokens and Retraining
Conference on Empirical Methods in Natural Language Processing · 2021
43
cited
Sotabase