Vitor Sousa

Search

Try a topic

↑ ↓ Navigate Enter OpenEsc Close

All series

Series · 5 parts

Adaptive Optimization at Scale: Contextual Bandits from Theory to Production

This series covers the full journey of contextual bandits — from knowing when to reach for them over A/B tests or full RL, through the mathematical foundations, algorithm implementations, neural extensions, and finally production deployment with offline evaluation and monitoring.

5 parts · ~113 min total reading time · Last updated Nov 2025

All five articles are published. Read in order, or use the prerequisites below to enter midway.

Start with the decision framework; the mathematical foundations introduce the notation used in later implementations. Algorithm code appears alongside each implementation.

  1. Part 1: When to Use Contextual Bandits: The Decision Framework

    When to use bandits vs A/B tests vs full RL. Build the mental model for choosing the right optimization approach.

    ~20 min decision framework · A/B testing · online learning

  2. Part 2: Contextual Bandit Theory: Regret Bounds and Exploration

    Regret bounds, reward models, and the exploration-exploitation tradeoff — the theory that makes bandit algorithms work.

    ~18 min regret bounds · reward models · exploration-exploitation

  3. Part 3: Implementing Contextual Bandits: Complete Algorithm Guide

    Epsilon-greedy, UCB, LinUCB, Thompson Sampling — implementations and tuning for each algorithm.

    ~25 min epsilon-greedy · UCB · LinUCB · Thompson Sampling

  4. Part 4: Neural Contextual Bandits for High-Dimensional Data

    Bootstrap uncertainty, embeddings, and two-stage selection for scaling bandits with neural networks.

    ~22 min neural networks · bootstrap · embeddings

  5. Part 5: Deploying Contextual Bandits: Production Guide and Offline Evaluation

    Systems design, offline evaluation, and monitoring for running contextual bandits in production.

    ~28 min production · offline evaluation · monitoring

Start with part 1