Skip to main content
Articles tagged with from-scratch | Vitor Sousa
Vitor Sousa
Writing
Projects
Foundations
Start Here
Bits
⌘K
Writing
Projects
Foundations
Start Here
Bits
Toggle theme
Suggested searches
transformers
evaluation
reinforcement learning
contextual bandits
Articles tagged from-scratch
Back to blog
→
Jul 24
A Transformer from Raw Tensors: What 112 Tests Taught Me
The model-foundations close-out of Learning with Verifiable Rewards: a decoder-only transformer built from raw tensors — attention with RoPE, RMSNorm, SwiGLU, an incremental KV-cache, four sampling modes — and the three properties that are easiest to get silently wrong. A case study in verification, not assembly.
~8 min