Sotabase

Career

· Summer Associate, WilmerHale2025–2025
· Legal Intern, Administrative Conference of the United States2024–2025
· AI Research Scientist, Meta2024–2025
· Research Intern at Microsoft Research New England, Microsoft2023–2023
· Machine Learning Engineer, OpenEvidence2023–2023
· Doctor of Law - JD, Stanford Law School2023–2026
· Graduate Student Fellow, Stanford RegLab2023–
· Summer Research Intern at Google Brain, Google DeepMind2022–2022
· PhD in Computer Science, Stanford University School of Engineering2021–2026
· Mathematics Instructor, Art of Problem Solving2020–2021
· Machine Learning Engineer, Scalable Magic (Maximus)2020–2021
· Teaching Fellow at HarvardX, edX2017–2019
· Bachelor of Arts - BA, Harvard University2016–2020

Publications (47)

Scaling Instruction-Finetuned Language Models
Journal of machine learning research · 2022
3,860
cited
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
arXiv.org · 2022
2,201
cited
Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them
Annual Meeting of the Association for Computational Linguistics · 2022
1,625
cited
Holistic Evaluation of Language Models
Trans. Mach. Learn. Res. · 2023
1,324
cited
Language Models are Multilingual Chain-of-Thought Reasoners
International Conference on Learning Representations · 2022
519
cited
Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructions
International Conference on Learning Representations · 2023
333
cited
Assessing the potential of GPT-4 to perpetuate racial and gender biases in health care: a model evaluation study.
The Lancet Digital Health · 2024
318
cited
Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools
Journal of Empirical Legal Studies · 2024
212
cited
Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models
Journal of Legal Analysis · 2024
171
cited
Meta-Prompting: Enhancing Language Models with Task-Agnostic Scaffolding
arXiv.org · 2024
123
cited
When Benchmarks are Targets: Revealing the Sensitivity of Large Language Model Leaderboards
Annual Meeting of the Association for Computational Linguistics · 2024
119
cited
A Benchmark for Learning to Translate a New Language from One Grammar Book
International Conference on Learning Representations · 2023
81
cited
LSTM Networks Can Perform Dynamic Counting
Proceedings of the Workshop on Deep Learning and Formal Languages: Building Bridges · 2019
80
cited
When Do Pre-Training Biases Propagate to Downstream Tasks? A Case Study in Text Summarization
Conference of the European Chapter of the Association for Computational Linguistics · 2023
76
cited
Prompt-and-Rerank: A Method for Zero-Shot and Few-Shot Arbitrary Textual Style Transfer with Small Language Models
Conference on Empirical Methods in Natural Language Processing · 2022
71
cited
Memory-Augmented Recurrent Neural Networks Can Learn Generalized Dyck Languages
arXiv.org · 2019
53
cited
Follow the Wisdom of the Crowd: Effective Text Generation via Minimum Bayes Risk Decoding
Annual Meeting of the Association for Computational Linguistics · 2022
52
cited
On Evaluating the Generalization of LSTM Models in Formal Languages
arXiv.org · 2018
47
cited
The Harvard USPTO Patent Dataset: A Large-Scale, Well-Structured, and Multi-Purpose Corpus of Patent Applications
Neural Information Processing Systems · 2022
39
cited
Coding Inequity: Assessing GPT-4's Potential for Perpetuating Racial and Gender Biases in Healthcare
medRxiv · 2023
33
cited
Sotabase
Mirac Suzgun | Researcher Profile | Sotabase | Sotabase