J. Wilder Lavington
Machine learning research on optimization, reinforcement learning, and generative models.
I am an Applied Scientist II at Amazon, working on automated vendor negotiation: reinforcement learning decides the strategy, causal inference estimates what a given term is actually worth, and language models handle the reading and writing. Previously I worked on explainable recommender systems and contact routing. I completed my PhD in Computer Science at the University of British Columbia in 2024.
My research is about optimization when the objective is a moving target. In reinforcement and imitation learning, gradients are expensive and the data distribution shifts as the policy changes, which breaks assumptions most optimizers are built on. I work on methods designed for that setting, on using the privileged state inside a simulator to train agents that only see part of the world, and on generative models of traffic realistic enough to test them against. This work has appeared at ICML, ICLR, and NeurIPS.
Outside work I climb, ski, cycle, and spend as many nights outside as I can.
Recently implemented
- Reinforcement learning
- Policy optimization (PPO, GRPO)
- Online imitation learning
- Off-policy RL and experience replay
- Asymmetric learning in POMDPs
- Planning as inference
- Self-play
- Language model post-training
- Supervised fine-tuning
- Preference optimization
- Parameter-efficient fine-tuning
- Reward modeling
- Constrained decoding
- Generative modeling
- Diffusion models
- Normalizing flows
- Sequential Monte Carlo
- Score estimation
- Consistency models
- Optimization
- Stochastic optimization
- Majorization-minimization surrogates
- Coordinate descent
- Adaptive gradient methods
- Negotiation and decision theory
- Alternating-offers bargaining
- Opponent modeling
- Utility elicitation
- Nash bargaining solution
Background
Full CV →- 2013 – 2017 Boulder, Colorado
BS, Applied Mathematics
University of Colorado Boulder
Concentration in computational statistics. First analytics work as an intern at Analytic Partners and Seagate Technology.
- 2017 – 2018 Boulder, Colorado
MS, Applied Mathematics
University of Colorado Boulder
Mathematical modeling. Thesis built a probabilistic model of CRISPR-Cas9 gene editing.
- 2018 – 2024 Vancouver, British Columbia
PhD, Computer Science
University of British Columbia
Optimization methods built for reinforcement and imitation learning, plus asymmetric learning from privileged simulator state. Applied science internship at Amazon in 2019.
- 2020 – 2024 Vancouver, British Columbia
Researcher · Inverted AI
Alongside the PhD
Behavior models and simulation for autonomous driving, including the TorchDriveEnv benchmark and planning-as-inference work on collision avoidance.
- 2024 – Present Vancouver, British Columbia
Applied Scientist II · Amazon
Current
Automated vendor negotiation, combining reinforcement learning, causal inference, and language models. Previously explainable recommender systems and contact routing.
Research
Optimization for reinforcement learning
Optimizers built for the structure of policy optimization, where gradients are expensive and objectives shift as the policy changes.
Imitation and asymmetric learning
Using privileged simulator state to train experts that a partially observed agent can actually imitate safely.
Generative models for driving
Diffusion models, normalizing flows, and sequential Monte Carlo for realistic, controllable multi-agent traffic behavior.
Automated negotiation
Multi-turn bargaining as sequential decision making: policies that choose offers, beliefs over the counterparty, and language models constrained to carry out the chosen action.
