The curriculum for learning the core RL+inference systems ideas is not long:
- trainer: hf ultra-scale playbook
- inference: Baseten inference engineering
- algorithms/recipes: Lambert’s RLHF book
- kernels: PMPP (foundation chapters suffice)
Blog posts fill the rest (below)
Conversation
PyTorch internals: blog.ezyang.com/2019/05/pytorc
vLLM internals: vllm.ai/blog/2025-09-0
Ray 2.0 architecture: docs.google.com/document/u/0/d
Collective communications: aleksagordic.com/blog/collectiv
Little bit stale review of the RL systems design space: huggingface.co/blog/async-rl-
Fighting
alphaXiv is telling a different story
Great resource! Is it intentional you didn't add
wow, amazing technical blog post about microsoft's new 3T pre-training with very rare and clear insights on how to train models of this size
first let's get an overview of the shape of this model, it's a sparser model than Kimi K3 (80B active vs 104B), same sliding window hybrid
Today, we're publishing two new findings from inside Anthropic: how work and hiring are changing at a company at the frontier of AI adoption, and how Anthropic staff expect AI to change the economy.