M5: Counterfactual Regret Minimization
Counterfactual Regret Minimization is the algorithm that made AI superhuman at poker, Texas Hold'em, and Hearthstone. We build from external and internal regret to the full CFR algorithm: decomposition, convergence guarantees, and implementation techniques. For LLM agents, CFR supplies a computational backbone for self-play training—the pattern by which agents improve by playing against themselves that recurs throughout LLM research. PA4 extends this thread into multi-agent debate and mechanism design.
Module preview
Counterfactual Regret Minimization is the algorithm that made AI superhuman at poker, Texas Hold'em, and Hearthstone. We build from external and internal regret to the full CFR algorithm: decomposition, convergence guarantees, and implementation techniques. For LLM agents, CFR supplies a computational backbone for self-play training—the pattern by which agents improve by playing against themselves that recurs throughout LLM research. PA4 extends this thread into multi-agent debate and mechanism design.
Lectures and materials
L11: Regret Minimization
External and internal regret, regret matching, and convergence.
L12: CFR Algorithm
Counterfactual decomposition and self-play in imperfect-information games.
Programming assignment
PA4: Multi-Agent Debate · Due November 21 at 11:59 PM CDT