Vitor Sousa

Search

Try a topic

↑ ↓ Navigate Enter OpenEsc Close

Essays, explainers, and notes

Writing

Longer pieces on machine-learning systems: derivations of known methods, notes from implementing them, and readings of papers. Each one is labelled by the kind of evidence it offers.

Start hereFoundationsAll seriesShorter notesRSS

Series and reading paths

Start here
Foundations · 3 parts

Transformer Internals

Attention, positional encoding, and the full forward pass, derived and then implemented with tests.

 Series · 5 parts

Reinforcement learning for LLMs

From policy gradients and PPO to GRPO, GDPO, and a map of the GRPO family.

 Series · 5 parts

Contextual bandits

When to use them, the theory, four algorithms side by side, and offline evaluation.

Archive

16 pieces
What the labels mean
Explainer
A synthesis or derivation of a known method. Needs: references, clear math or intuition, and correct attribution.
Engineering note
An implementation decision, bug, test, or performance trade-off. Needs: code or a reproducible example, and the exact conditions.
Paper note
A reading and critique of one paper. Needs: the reference, its core claim, limitations, and my interpretation.

Engineering note

A Transformer from Raw Tensors: What 112 Tests Taught Me

What it took to verify a decoder-only transformer built from raw tensors, and the three properties that were easiest to get silently wrong.

8 min

Explainer · RL for LLMs · Part 5

After GDPO: A Map of the GRPO Family

Reading Dr. GRPO, DAPO, GSPO and GDPO as points along seven design axes of GRPO, with a guide to choosing between them.

18 min

Explainer · Transformer Internals · Part 3

Building a Transformer: The Complete Forward Pass

Normalization, residual connections, feed-forward layers, and a decoder-only transformer assembled piece by piece with tested code.

35 min

Explainer · Transformer Internals · Part 2

Positional Encoding: Teaching Transformers to Count

Why a transformer needs position information, and how sinusoidal, learned, and rotary embeddings each encode it.

30 min

Explainer · Transformer Internals · Part 1

Attention Is All You Need to Implement

Scaled dot-product and multi-head attention derived from the matrix math, then implemented and tested in PyTorch.

40 min

Explainer · RL for LLMs · Part 4

GDPO: Multi-Reward RL Done Right

Why summing rewards before GRPO's group normalization can wash out the smaller ones, and how normalizing each reward separately avoids it.

25 min

Explainer · RL for LLMs · Part 3

GRPO: Eliminating the Value Network

How GRPO replaces PPO's learned value function with group-relative advantages, and what that removes from the training setup.

32 min

Explainer · RL for LLMs · Part 2

PPO for Language Models: The RLHF Workhorse

PPO's trust region, clipped objective and GAE, worked through for language models, and why its four-model setup becomes expensive at scale.

28 min

Explainer · RL for LLMs · Part 1

Reinforcement Learning Foundations for LLM Alignment

Notes on the RL foundations behind LLM post-training: MDPs, policy gradients, value functions, and actor-critic methods.

35 min

Explainer · Contextual bandits · Part 5

Deploying Contextual Bandits: Production Guide and Offline Evaluation

Systems design, offline evaluation, and monitoring for running a contextual bandit safely once it leaves the notebook.

24 min

Explainer · Contextual bandits · Part 4

Neural Contextual Bandits for High-Dimensional Data

When do neural contextual bandits become useful? Bootstrap ensembles for uncertainty, and embeddings for large action spaces.

22 min

Explainer · Contextual bandits · Part 3

Implementing Contextual Bandits: Complete Algorithm Guide

Implementing and comparing four contextual bandit algorithms in Python, and what decides between them in practice.

25 min

Explainer · Contextual bandits · Part 2

Contextual Bandit Theory: Regret Bounds and Exploration

What regret bounds and the exploration–exploitation trade-off actually say, and how they inform the choice of algorithm.

18 min

Explainer · Contextual bandits · Part 1

When to Use Contextual Bandits: The Decision Framework

When an adaptive, personalized policy is worth more than a fixed A/B test, and when the simpler option is the right one.

20 min

Explainer

Beyond the Vibe Check: A Systematic Approach to LLM Evaluation

What makes an LLM evaluation trustworthy, and which parts of a pipeline — metrics, LLM judges, bias checks, monitoring — earn that trust.

60 min

Paper note

OpenELM Notes

Reading Apple's OpenELM paper: layer-wise parameter scaling and an openly released training and evaluation framework.

8 min