If you want to know what a system is actually for, don’t read its mission statement. Read its incentives. A system does what it rewards, reliably and without apology, no matter what it claims to value.
The gap between stated and structural
Every organization has two constitutions. The stated one lives in documents. The structural one lives in what gets measured, funded, promoted, and punished. When the two disagree, the structural one wins every time — not because people are cynical, but because incentives are the slope of the ground everyone walks on.
This is why load-bearing values matter. A value that has not been wired into an incentive is not yet real; it is a preference, waiting for the first hard tradeoff to reveal that it was never structural at all.
AI makes incentives more powerful, faster
Automated systems execute incentives with inhuman fidelity and speed. A human following a bad metric will eventually feel the wrongness and fudge it. An optimizer will pursue the metric straight off a cliff, because the metric is its world. As we hand more decisions to systems that optimize, the cost of a badly drawn incentive rises sharply.
Designing the slope
A few working rules: measure the outcome, not the proxy; make the good path the cheap path, so doing right doesn’t require heroism; and always build in a way to be wrong, because an incentive with no feedback loop is a bet you can never update. The page beneath this one takes the first rule apart in detail — because every proxy is eventually gamed.