AI Concepts · Goodhart's Law

The Rating Game

Rate 8 pairs of answers. Your picks train a reward model. Then an optimizer writes for it, and you decide how hard to let it push.

1
Rate
Pick the better answer in each of 8 pairs. A and B on a keyboard work too.
2
Fit
Your picks become a reward model, fit in front of you from two things it can measure: how long an answer is and how sure it sounds.
3
Ship
Turn up the optimizer, spend up to 2 spot checks on the true quality, and ship. Only then do you see what it cost.

Related

Theme
Trust
Both demonstrate Goodhart's Law - how measurement systems corrupt authentic behavior
Theory
The New Sorting Hat
Both examine how AI classification systems create systematic biases through optimization pressure
Theme
The Invisible Architecture
Both show how measurement systems become invisible infrastructure that shapes behavior
Theme
Subgenre Survival
Both explore tension between authentic expression and external validation/success metrics