LLM behavioral simulation
Applied Research Scientist, Openstream.aiPh.D., University of Florida
I build language-model simulations of human behavior that hold up at the population level, so that researchers can run studies on simulated people when recruiting real ones is the bottleneck.
My PhD in Computer Science at the University of Florida, advised by Kristy Boyer and Bonnie Dorr, developed generative agent-based models of learners and calibrated them to population-level behavioral targets through test-time compute allocation.
My current work trains models against those behavioral targets directly with reinforcement learning. Before and during the PhD, I worked on task-oriented dialogue and synthetic data generation at Amazon Alexa, Openstream.ai, and Rasa.
A simulated population is useful as a scientific instrument only if its aggregate behavior matches what real populations do. My work treats that match as the objective: I measure where LLM simulations of people diverge from human behavior, calibrate them at inference time, and train models so that the calibration becomes a property of the model.
Role-playing agents can describe how a persona will behave and then simulate it doing something else. In LLM simulations of human trust, models' stated predictions about a persona do not reliably match the behavior they generate for it, so a simulation's self-report cannot substitute for behavioral validation.
My dissertation holds the generator and the target specification fixed and asks how inference-time compute should be distributed across a simulated population. Across fifty behavioral targets for simulated pair programming, whether a target is defined per session or over the whole batch predicts both its difficulty and which allocation strategy works.
HYP-MIX lets education researchers specify learner behavior as combinations of testable hypotheses and checks whether LLM-simulated learners stay behaviorally calibrated when those hypotheses are modified. It is the framework the learner-simulation work in my dissertation builds on.
Search at inference time can make a population hit its targets, but it pays that cost on every run. My current work moves the objective into training and uses reinforcement learning to optimize models directly against population-level behavioral targets.
Large Language Models for Whole-Learner Support: Opportunities and Challenges
A. Mannekote, A. Davies, J. D. Pinto, S. Zhang, D. Olds, N. L. Schroeder, B. Lehman, D. Zapata-Rivera, C. Zhai. Frontiers in Artificial Intelligence 7, 2024.
Towards Compositionally Generalizable Semantic Parsing in Large Language Models: A Survey
A. Mannekote. arXiv:2404.13074.
X. Tian, A. Mannekote, C. E. Solomon, Y. Song, C. F. Wise, T. McKlin, J. Barrett, K. E. Boyer, M. Israel. Educational Data Mining (EDM), 2024.
Can Similarity-Based Domain-Ordering Reduce Catastrophic Forgetting for Intent Recognition?
A. Mannekote, X. Tian, K. E. Boyer, B. J. Dorr. arXiv:2402.14155.
A. Mannekote, M. Celepkolu, J. B. Wiggins, K. E. Boyer. ACM Conference on Conversational User Interfaces (CUI), 2023.
Agreement Tracking for Multi-Issue Negotiation Dialogues
A. Mannekote, B. J. Dorr, K. E. Boyer. arXiv:2307.06524. Dataset
Towards a Neural Era in Dialogue Management for Collaboration: A Literature Survey
A. Mannekote. arXiv:2307.09021.
Don't Just Paste Your Stacktrace: Shaping Discussion Forums in Introductory CS Courses
A. Mannekote, M. Celepkolu, A. C. Galdo, K. E. Boyer, M. Israel, S. Heckman, K. Stephens-Martinez. ACM Technical Symposium on Computer Science Education (SIGCSE), 2022.
Applied Research Scientist, Openstream.ai
Graduate Research Assistant, University of Florida
Gainesville, FLResearch Intern, Amazon Alexa · MENTOR, in-context adaptation to feedback and tool-use instructions
Bellevue, WAResearch Intern, Openstream.ai · LLM semantic parsing for neuro-symbolic dialogue
RemoteResearch Intern, Amazon Alexa · IndirectRequests
Bellevue, WAData Scientist, Fractal Analytics & Crux Intelligence · Text-to-SQL evaluation
BengaluruMachine Learning Solutions Architect, Rasa
BerlinResearch Assistant, Robert Bosch Centre for Cyber-Physical Systems, Indian Institute of Science
BengaluruIntern, IBM Watson Machine Learning
BengaluruResearch Intern, Stride.AI
BengaluruPh.D., Computer Science, University of Florida · Neurosymbolic Authoring and Test-Time Compute Allocation for Generative Educational Simulations
Gainesville, FLB.E., Computer Science & Engineering, Ramaiah Institute of Technology
BengaluruPh.D. in Computer Science conferred by the University of Florida for the dissertation Neurosymbolic Authoring and Test-Time Compute Allocation for Generative Educational Simulations.
Joined Openstream.ai as an Applied Research Scientist to train models against behavioral targets.
Belief-behavior consistency in role-playing agents appeared at COLM 2025.
HYP-MIX appeared at AAAI 2025.
IndirectRequests appeared at COLING 2025, with the dataset released on Hugging Face.
HYP-MIX · authoring and calibration framework for LLM learner simulations
IndirectRequests · indirect user requests for schema-guided dialogue
GPT-NegoChat · multi-issue negotiation dialogues with agreement-state annotations
nonechucks · PyTorch datasets and samplers that skip invalid samples
I also design and build personal software with coding agents, such as Ratchet, an accountability timer with a cloud-synced goal tree.
Personal software