Reinforcement Learning

Eternal Term 5 of 7

Learning by doing — decisions, rewards, and consequences

What It Means

Reinforcement Learning (RL) is the study of how agents learn to make sequences of decisions by acting in an environment and receiving rewards or penalties. Rather than learning from labeled examples, RL agents learn from the consequences of their own actions — the same way humans and animals learn skills.

Why It Is Eternal

RL formalizes the oldest learning problem there is: how to act in an uncertain world to maximize long-term benefit. Prediction tells you what is true; RL tells you what to do. Planning, control, game-playing, and goal pursuit are permanent problems, and RL is their mathematics.

It has repeatedly delivered superhuman results that no other approach could: backgammon in the 1990s, Go in 2016, and modern game-playing and robotics systems since. Crucially, RL is now inside the LLM pipeline itself — RLHF (reinforcement learning from human feedback) is what turns raw language models into helpful, aligned assistants.

As AI moves from answering questions to taking actions — agents, robots, autonomous operations — the action-learning loop of RL becomes the operating system of that shift.

Core Ideas

Agent, environment, reward: The RL loop: an agent observes state, takes an action, and receives a reward and a new state. Everything else — algorithms, architectures, theory — serves this loop.
Exploration vs. exploitation: Should the agent try something new or repeat what worked? This trade-off is the heart of learning under uncertainty, and it shows up everywhere from A/B testing to robotics.
Value and policy: Value functions estimate how good a situation is; policies map situations to actions. Modern methods learn both, often with deep networks as function approximators.
RLHF and alignment: Human preferences become reward signals. RLHF fine-tunes language models toward helpfulness and safety, making RL a core production technique, not a lab curiosity.

Where It Shows Up

RLHF alignment of large language models and chat assistants
Game-playing superhuman agents: Go, chess, poker, and real-time strategy
Robotics: locomotion, manipulation, and warehouse automation
Operations optimization: datacenter cooling, traffic signals, pricing, and inventory

Milestones Through Time

1989 — Watkins introduces Q-learning, making model-free RL practical.
2016 — DeepMind's AlphaGo defeats Lee Sedol, showcasing learned intuition in an open-ended game.
2017 — AlphaZero masters chess, shogi, and Go from self-play alone — no human games.
2022 — RLHF takes center stage as the alignment technique behind ChatGPT-class models.

The Road Ahead

RL is becoming the glue between large models and real-world action. Agents that plan, call tools, and self-correct are implicitly doing RL even when trained otherwise. Expect hybrid systems: foundation models provide priors and language, while RL provides the decision discipline. Safe exploration — learning without expensive failures — remains the frontier.

The Takeaway

Whenever AI must act rather than just answer, RL is the framework. Learn the loop: state, action, reward, update — it recurses through everything from robotics to model alignment.