Curriculum Vitae

Applied Scientist at Amazon training reinforcement learning agents for automated negotiation. PhD in Computer Science, University of British Columbia (2024).

Education

• 2018 – 2024 PhD, Computer Science University of British Columbia · Vancouver, BC
• 2017 – 2018 MS, Applied Mathematics University of Colorado Boulder
• 2013 – 2017 BS, Applied Mathematics University of Colorado Boulder

Experience

• 2024 – Present Applied Scientist II · Amazon Automated vendor negotiation: reinforcement learning for strategy, causal inference for valuing terms, and LLMs for parsing and generating correspondence. Previously explainable recommender systems and contact routing. Managers: Amin Banitalebi, Peng Dai.
• 2020 – 2024 Researcher · Inverted AI Behavior models and simulation for autonomous driving. Supervisor: Frank Wood.
• 2019 Applied Scientist Intern · Amazon Supervisor: Amber Roy Chowdhury.
• 2017 Data Scientist Intern · Seagate Technology Two terms, spring and summer. Supervisor: Michael Renella.
• 2016 Analyst Intern · Analytic Partners Supervisor: Michael Leichman.

Teaching

• 2019 Machine Learning and Data Mining (CPSC 340) Teaching Assistant · University of British Columbia, Department of Computer Science
• 2018 Precalculus for Engineers (APPM 1235) Teaching Assistant · University of Colorado Boulder, Department of Applied Mathematics
• 2017 Calculus 3 for Engineers (APPM 2350) Teaching Assistant · University of Colorado Boulder, Department of Applied Mathematics
• 2016 Matrix Methods and Applications (APPM 3310) Learning Assistant · University of Colorado Boulder, Department of Applied Mathematics
• 2015 Intermediate Numerical Analysis (APPM 4650) Learning Assistant · University of Colorado Boulder, Department of Applied Mathematics
• 2014 Applied Probability (APPM 3570) Learning Assistant · University of Colorado Boulder, Department of Applied Mathematics

Skills

• Research areas
  • Optimization for machine learning
  • Reinforcement learning
  • Imitation learning
  • Generative modeling
  • Probabilistic inference
  • Automated negotiation
  • Causal inference
  • LLM post-training
  • Recommender systems
• Reinforcement and imitation learning
  • Policy optimization (PPO, GRPO)
  • Off-policy reinforcement learning
  • Soft actor-critic
  • Experience replay and prioritized sampling
  • Continuous control
  • Online imitation learning (DAgger)
  • Behavior cloning
  • Asymmetric learning from privileged state
  • POMDPs
  • Reinforcement learning as inference
  • Planning as inference
  • Reward modeling
  • Self-play
• Optimization
  • Stochastic optimization
  • Majorization-minimization surrogates
  • Coordinate descent
  • Equality- and bound-constrained optimization
  • Steepest descent in general norms
  • Adaptive gradient methods (Adam, sign descent)
  • Online convex optimization
  • Follow-the-regularized-leader
  • Convergence and regret analysis
  • Polyak-Lojasiewicz conditions
• Generative and probabilistic models
  • Diffusion models
  • Conditional diffusion
  • Score estimation
  • Consistency models
  • Probability-flow ODEs
  • Continuous normalizing flows
  • Permutation-invariant models
  • Sequential Monte Carlo
  • Variance reduction
  • Divergence minimization
  • Markov chain models
• Language models
  • Supervised fine-tuning
  • Preference optimization
  • Parameter-efficient fine-tuning (LoRA, QLoRA)
  • Constrained decoding
  • Distributed GPU training
  • Inference serving
• Simulation and applied domains
  • Autonomous driving simulation
  • Multi-agent trajectory prediction
  • Behavior foundation models
  • Differentiable simulators
  • Reinforcement learning benchmarks
  • Synthetic data and distribution shift
  • 3D object detection
  • Video inpainting
  • Collision avoidance
• Languages
  • Python
  • C/C++
  • SQL (Hive, Oracle)
  • R
  • MATLAB
  • Clojure
  • Erlang
• Frameworks and libraries
  • PyTorch
  • Hugging Face Transformers
  • PEFT
  • TRL
  • vLLM
  • Accelerate
  • bitsandbytes
  • Ray
  • TensorFlow
  • Keras
  • NumPy
  • SciPy
  • scikit-learn
  • pandas
  • NetworkX
  • PySpark
• Tools and platforms
  • Linux
  • Git
  • Docker
  • AWS SageMaker
  • Amazon Bedrock
  • S3
  • Athena
  • Redshift
  • LaTeX
  • Hadoop
  • Tableau
  • Mathematica

Publications

18 papers, preprints, and theses, including work at ICML, ICLR, NeurIPS, IROS, and IEEE ITSC. See the full list or Google Scholar.

Taking Advantage of Common Assumptions in Policy Optimization and Reinforcement Learning

J. Wilder Lavington

PhD Thesis 2024 PhD Thesis, University of British Columbia

• Paper
• Summary
• BibTeX

Nearest Neighbour Score Estimators for Diffusion Generative Models

Matthew Niedoba, Dylan Green, Saeid Naderiparizi, Vasileios Lioutas, J. Wilder Lavington, Xiaoxuan Liang, Yunpeng Liu, Ke Zhang, Setareh Dabiri, Adam Scibior, Berend Zwartsenberg, Frank Wood

ICML 2024 International Conference on Machine Learning (ICML)

• arXiv
• Summary
• BibTeX

TorchDriveEnv: A Reinforcement Learning Benchmark for Autonomous Driving with Reactive, Realistic, and Diverse Non-Playable Characters

J. Wilder Lavington, Ke Zhang, Vasileios Lioutas, Matthew Niedoba, Yunpeng Liu, Dylan Green, Saeid Naderiparizi, Xiaoxuan Liang, Setareh Dabiri, Adam Scibior, Berend Zwartsenberg, Frank Wood

arXiv 2024 arXiv preprint

• arXiv
• Summary
• BibTeX

A Diffusion-Model of Joint Interactive Navigation

Matthew Niedoba, J. Wilder Lavington, Yunpeng Liu, Vasileios Lioutas, Justice Sefas, Xiaoxuan Liang, Dylan Green, Setareh Dabiri, Berend Zwartsenberg, Adam Scibior, Frank Wood

NeurIPS 2023 Advances in Neural Information Processing Systems (NeurIPS)

• Paper
• Summary
• BibTeX

Noise Is Not the Main Factor Behind the Gap Between SGD and Adam on Transformers, But Sign Descent Might Be

Frederik Kunstner, Jacques Chen, J. Wilder Lavington, Mark Schmidt

ICLR 2023 International Conference on Learning Representations (ICLR)

• Paper
• Summary
• BibTeX

Target-Based Surrogates for Stochastic Optimization

J. Wilder Lavington, Sharan Vaswani, Reza Babanezhad, Mark Schmidt, Nicolas Le Roux

ICML 2023 International Conference on Machine Learning (ICML)

• PDF
• Summary
• BibTeX

Critic Sequential Monte Carlo

Vasileios Lioutas, J. Wilder Lavington, Justice Sefas, Matthew Niedoba, Yunpeng Liu, Berend Zwartsenberg, Setareh Dabiri, Frank Wood, Adam Scibior

ICLR 2023 International Conference on Learning Representations (ICLR)

• Paper
• Summary
• BibTeX

Improved Policy Optimization for Online Imitation Learning

J. Wilder Lavington, Sharan Vaswani, Mark Schmidt

CoLLAs 2022 Conference on Lifelong Learning Agents (CoLLAs)

• PDF
• Summary
• BibTeX

Robust Asymmetric Learning in POMDPs

Andrew Warrington, J. Wilder Lavington, Adam Scibior, Mark Schmidt, Frank Wood

ICML 2021 International Conference on Machine Learning (ICML)

• Paper
• Summary
• BibTeX