Sotabase

Career

· Assistant Professor in Computer Science, Princeton University2024–
· Co-founder, Chief Scientist, Together AI2023–
· MS in Statistics, Stanford University2018–2019
· PhD in Computer Science, Stanford University2016–2023
· MS in Computer Science, Stanford University2014–2016
· BS in Mathematics, Stanford University2012–2016

Publications (83)

FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
Neural Information Processing Systems · 2022
3,505
cited
FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning
International Conference on Learning Representations · 2023
2,186
cited
StarCoder: may the source be with you!
Trans. Mach. Learn. Res. · 2023
1,079
cited
Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layers
Neural Information Processing Systems · 2021
969
cited
HiPPO: Recurrent Memory with Optimal Polynomial Projections
Neural Information Processing Systems · 2020
831
cited
Hungry Hungry Hippos: Towards Language Modeling with State Space Models
International Conference on Learning Representations · 2022
567
cited
StarCoder 2 and The Stack v2: The Next Generation
arXiv.org · 2024
561
cited
Hyena Hierarchy: Towards Larger Convolutional Language Models
International Conference on Machine Learning · 2023
437
cited
Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time
International Conference on Machine Learning · 2023
285
cited
S4ND: Modeling Images and Videos as Multidimensional Signals with State Spaces
Neural Information Processing Systems · 2022
225
cited
A Kernel Theory of Modern Data Augmentation
International Conference on Machine Learning · 2018
208
cited
Scatterbrain: Unifying Sparse and Low-rank Attention Approximation
Neural Information Processing Systems · 2021
153
cited
Decentralized Training of Foundation Models in Heterogeneous Environments
Neural Information Processing Systems · 2022
125
cited
Learning Fast Algorithms for Linear Transforms Using Butterfly Factorizations
International Conference on Machine Learning · 2019
123
cited
Monarch: Expressive Structured Matrices for Efficient and Accurate Training
International Conference on Machine Learning · 2022
116
cited
Pixelated Butterfly: Simple and Efficient Sparse training for Neural Network Models
International Conference on Learning Representations · 2021
91
cited
Effectively Modeling Time Series with Simple Discrete State Spaces
International Conference on Learning Representations · 2023
71
cited
Simple Hardware-Efficient Long Convolutions for Sequence Modeling
International Conference on Machine Learning · 2023
66
cited
MONGOOSE: A Learnable LSH Framework for Efficient Neural Network Training
International Conference on Learning Representations · 2021
63
cited
Kaleidoscope: An Efficient, Learnable Representation For All Structured Linear Maps
International Conference on Learning Representations · 2020
54
cited
Sotabase
Tri Dao | Researcher Profile | Sotabase | Sotabase