Sotabase
Home
Researchers
Career
·
Research Intern
,
Berkeley Artificial Intelligence Research (BAIR)
2021–
Publications
(20)
White-Box Transformers via Sparse Rate Reduction
Neural Information Processing Systems · 2023
122
cited
Active-Dormant Attention Heads: Mechanistically Demystifying Extreme-Token Phenomena in LLMs
arXiv.org · 2024
28
cited
Token Statistics Transformer: Linear-Time Attention via Variational Rate Reduction
arXiv.org · 2024
27
cited
White-Box Transformers via Sparse Rate Reduction: Compression Is All There Is?
arXiv.org · 2023
27
cited
Independent and Decentralized Learning in Markov Potential Games
IEEE Transactions on Automatic Control · 2022
26
cited
Emergence of Segmentation with Minimalistic White-Box Transformers
CPAL · 2023
25
cited
Masked Completion via Structured Diffusion with White-Box Transformers
International Conference on Learning Representations · 2024
16
cited
Representation Learning via Manifold Flattening and Reconstruction
Journal of machine learning research · 2023
15
cited
Scaling White-Box Transformers for Vision
Neural Information Processing Systems · 2024
14
cited
A Global Geometric Analysis of Maximal Coding Rate Reduction
International Conference on Machine Learning · 2024
11
cited
Simplifying DINO via Coding Rate Regularization
International Conference on Machine Learning · 2025
11
cited
Congestion Pricing for Efficiency and Equity: Theory and Applications to the San Francisco Bay Area
arXiv.org · 2024
7
cited
Closed-Loop Transcription via Convolutional Sparse Coding
arXiv.org · 2023
6
cited
Pursuit of a Discriminative Representation for Multiple Subspaces via Sequential Games
Journal of the Franklin Institute · 2022
6
cited
On the Edge of Memorization in Diffusion Models
arXiv.org · 2025
5
cited
Attention-Only Transformers via Unrolled Subspace Denoising
International Conference on Machine Learning · 2025
4
cited
Learning Low-Dimensional Structure via Closed-Loop Transcription: Equilibria and Optimization
2023
Note 14: Symmetric Matrices and Spectral Theorem
Sotabase
Druv Pai | Researcher Profile | Sotabase | Sotabase