J. Wilder Lavington
Reinforcement learning, optimization, and generative modeling applied to real-world problems.
I am an Applied Scientist II at Amazon, where I train reinforcement learning agents to negotiate. Vendor negotiation is a sequential decision problem: a policy trained with self-play chooses each offer and concession, causal inference estimates what a given term is actually worth, and language models turn the chosen action into correspondence. The policy decides; the LLM speaks. Previously I worked on explainable recommender systems and contact routing. I completed my PhD in Computer Science at the University of British Columbia in 2024.
My PhD was about learning policies in simulation, where training is cheaper and safer than the real world, and where the simulator knows things the agent cannot see — the exact positions of other cars, not just a camera view. The thesis tackles four pieces of that problem: how online imitation learning behaves as the policy class becomes more expressive, optimizers built for policy optimization where every gradient is expensive to query, using privileged simulator state to train experts that a partially observed agent can safely imitate, and generating driving behavior that satisfies hard constraints with sequential Monte Carlo.
Outside work I climb, ski, cycle, and spend as many days outside as I can.
Recently implemented
- Reinforcement learning
- Policy optimization (PPO, GRPO)
- Online imitation learning
- Off-policy RL and experience replay
- Asymmetric learning in POMDPs
- Planning as inference
- Self-play
- Language model post-training
- Supervised fine-tuning
- Preference optimization
- Parameter-efficient fine-tuning
- Reward modeling
- Constrained decoding
- Generative modeling
- Diffusion models
- Normalizing flows
- Sequential Monte Carlo
- Score estimation
- Consistency models
- Optimization
- Stochastic optimization
- Majorization-minimization surrogates
- Coordinate descent
- Adaptive gradient methods
- Negotiation and decision theory
- Alternating-offers bargaining
- Opponent modeling
- Utility elicitation
- Nash bargaining solution
Background
Full CV →- 2013 – 2017 Boulder, Colorado
BS, Applied Mathematics
University of Colorado Boulder
Concentration in computational statistics. First analytics work as an intern at Analytic Partners and Seagate Technology.
- 2017 – 2018 Boulder, Colorado
MS, Applied Mathematics
University of Colorado Boulder
Mathematical modeling. Thesis built a probabilistic model of CRISPR-Cas9 gene editing.
- 2018 – 2024 Vancouver, British Columbia
PhD, Computer Science
University of British Columbia
Optimization methods built for reinforcement and imitation learning, plus asymmetric learning from privileged simulator state. Applied science internship at Amazon in 2019.
- 2020 – 2024 Vancouver, British Columbia
Researcher · Inverted AI
Alongside the PhD
Behavior models and simulation for autonomous driving, including the TorchDriveEnv benchmark and planning-as-inference work on collision avoidance.
- 2024 – Present Vancouver, British Columbia
Applied Scientist II · Amazon
Current
Reinforcement learning agents for vendor negotiation, supported by causal inference and language models. Previously explainable recommender systems and contact routing.
Research
Optimization for reinforcement learning
Optimizers built for the structure of policy optimization, where gradients are expensive and objectives shift as the policy changes.
Imitation and asymmetric learning
Using privileged simulator state to train experts that a partially observed agent can actually imitate safely.
Generative models for driving
Diffusion models, normalizing flows, and sequential Monte Carlo for realistic, controllable multi-agent traffic behavior.
Automated negotiation
Multi-turn bargaining as sequential decision making: policies that choose offers, beliefs over the counterparty, and language models constrained to carry out the chosen action.
