Sotabase

Career

· Junior Portfolio Manager, Goldman Sachs
· Research Intern, Google DeepMind
· Master's Student in Statistics, Stanford University
· PhD in Computer Science, Stanford University
· Researcher, Thinking Machines
· Research Scientist, Thinking Machines
· Undergraduate Student, UC Berkeley

Publications (74)

Direct Preference Optimization: Your Language Model is Secretly a Reward Model
Neural Information Processing Systems · 2023
7,064
cited
OpenVLA: An Open-Source Vision-Language-Action Model
Conference on Robot Learning · 2024
1,478
cited
Open X-Embodiment: Robotic Learning Datasets and RT-X Models : Open X-Embodiment Collaboration0
IEEE International Conference on Robotics and Automation · 2023
785
cited
Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback
Conference on Empirical Methods in Natural Language Processing · 2023
547
cited
Diffusion Model Alignment Using Direct Preference Optimization
Computer Vision and Pattern Recognition · 2023
542
cited
COMBO: Conservative Offline Model-Based Policy Optimization
Neural Information Processing Systems · 2021
486
cited
Open X-Embodiment: Robotic Learning Datasets and RT-X Models : Open X-Embodiment Collaboration
IEEE International Conference on Robotics and Automation · 2024
282
cited
From $r$ to $Q^*$: Your Language Model is Secretly a Q-Function
2024
230
cited
Disentangling Length from Quality in Direct Preference Optimization
Annual Meeting of the Association for Computational Linguistics · 2024
184
cited
Preference Fine-Tuning of LLMs Should Leverage Suboptimal, On-Policy Data
International Conference on Machine Learning · 2024
176
cited
Aligning Modalities in Vision Large Language Models via Preference Fine-tuning
arXiv.org · 2024
175
cited
Agent Q: Advanced Reasoning and Learning for Autonomous AI Agents
arXiv.org · 2024
151
cited
Offline Reinforcement Learning from Images with Latent Space Models
Conference on Learning for Dynamics & Control · 2020
142
cited
Offline Meta-Reinforcement Learning with Advantage Weighting
International Conference on Machine Learning · 2020
119
cited
Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data
arXiv.org · 2024
108
cited
Scaling Laws for Reward Model Overoptimization in Direct Alignment Algorithms
Neural Information Processing Systems · 2024
102
cited
Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thought
arXiv.org · 2025
85
cited
Generative Reward Models
arXiv.org · 2024
83
cited
Contrastive Preference Learning: Learning from Human Feedback without RL
arXiv.org · 2023
74
cited
An Emulator for Fine-Tuning Large Language Models using Small Language Models
International Conference on Learning Representations · 2023
66
cited
Sotabase
Rafael Rafailov | Researcher Profile | Sotabase | Sotabase