Sotabase

Career

· Principal Reseacher, Microsoft2025–
· Staff Research Scientist, Apple2024–2025
· Senior Research Scientist, Apple2022–2024
· Research Scientist, Apple2020–2022
· Research Intern, Microsoft2019–2019
· Research Intern, Google2018–2018
· Creative Technologies Lab Research Intern, Adobe2017–2017
· Research Intern, Adobe2016–2017
· Graduate Research Assistant, University of Washington2015–2019
· Software Engineer II, Microsoft2014–2015
· Software Development Engineer In Test, Microsoft2012–2012
· Software Engineer, Microsoft2012–2014
· Graduate Research Assistant, University of Nebraska-Lincoln2010–2012

Publications (24)

Screen Recognition: Creating Accessibility Metadata for Mobile Applications from Pixels
International Conference on Human Factors in Computing Systems · 2021
195
cited
Ferret-UI: Grounded Mobile UI Understanding with Multimodal LLMs
European Conference on Computer Vision · 2024
157
cited
Scout: Rapid Exploration of Interface Layout Alternatives through High-Level Design Constraints
International Conference on Human Factors in Computing Systems · 2020
72
cited
AXNav: Replaying Accessibility Tests from Natural Language
International Conference on Human Factors in Computing Systems · 2023
70
cited
Rewire: Interface Design Assistance from Examples
International Conference on Human Factors in Computing Systems · 2018
67
cited
Towards Complete Icon Labeling in Mobile Applications
International Conference on Human Factors in Computing Systems · 2022
46
cited
Understanding Screen Relationships from Screenshots of Smartphone Applications
International Conference on Intelligent User Interfaces · 2022
31
cited
Professional
2000
29
cited
ILuvUI: Instruction-tuned LangUage-Vision modeling of UIs from Machine Conversations
International Conference on Intelligent User Interfaces · 2023
25
cited
Never-ending Learning of User Interfaces
ACM Symposium on User Interface Software and Technology · 2023
24
cited
UIClip: A Data-driven Model for Assessing User Interface Design
ACM Symposium on User Interface Software and Technology · 2024
21
cited
Towards Automated Accessibility Report Generation for Mobile Apps
ACM Trans. Comput. Hum. Interact. · 2023
14
cited
Misty: UI Prototyping Through Interactive Conceptual Blending
International Conference on Human Factors in Computing Systems · 2024
12
cited
Genie: Input Retargeting on the Web through Command Reverse Engineering
International Conference on Human Factors in Computing Systems · 2017
11
cited
Extracting Replayable Interactions from Videos of Mobile App Usage
arXiv.org · 2022
10
cited
Screen Correspondence: Mapping Interchangeable Elements between UIs
arXiv.org · 2023
9
cited
Automated Code Fix Suggestions for Accessibility Issues in Mobile Apps
arXiv.org · 2024
7
cited
From Interaction to Impact: Towards Safer AI Agent Through Understanding and Evaluating Mobile UI Operation Impacts
International Conference on Intelligent User Interfaces · 2024
7
cited
Scout: Mixed-Initiative Exploration of Design Variations through High-Level Design Constraints
ACM Symposium on User Interface Software and Technology · 2018
5
cited
From Interaction to Impact: Towards Safer AI Agents Through Understanding and Evaluating UI Operation Impacts
arXiv.org · 2024
4
cited
Sotabase
Amanda Swearngin | Researcher Profile | Sotabase | Sotabase