Why static benchmarks miss behavior under incentive

A static benchmark is a set of questions with known answers. The model reads each one, produces an output, and a scorer compares the output to the key. The score is a property of the model and the questions. Nothing in the setup depends on what the model gains or loses by answering one way rather than another.

Real deployments are not like that. An agent advising a person has a user who pushes back and a counterparty with its own goals. It has a history that shapes what it expects. Often it has a reward signal tuned on whether people were satisfied. Those are incentives, even when nobody designed them as such. Behavior that only appears under incentive cannot be observed by a procedure that removes incentive.

Deference is the clearest example. Give a model a question, let it answer correctly, then have an authority figure assert the wrong answer. Many models change their answer. On a static benchmark they were right. In the second turn they were wrong, and the benchmark had no second turn. The failure is not knowledge. It is the policy the model follows when knowledge and social pressure conflict.

Anchoring works the same way. A model asked to value a claim produces one number when the question is neutral and another when the prompt mentions a figure first. The static test asks the neutral question. The deployed system never gets a neutral question, because the other side always speaks first.

Over-persuasion is harder to see and more consequential. A model that has learned that confident, fluent answers are rated well will produce confident, fluent answers when it should hedge. Measured against a key, confidence is invisible. Measured against a person deciding whether to settle, confidence is the whole effect.

The experimental economics literature spent decades building procedures for exactly this problem with people. Put subjects in an environment with a payoff, vary one thing, and measure what they do, not what they say they would do. The environments are simple by design, so that the behavior can be attributed to the variable that changed. That discipline transfers directly to agents.

An environment for agents needs four properties. Incentives, so there is something to gain by behaving one way. Scarcity, so choices cost something. Changing information, so the agent has to update rather than recite. And other agents, so strategic behavior has a target. With those in place, the behaviors that static tests cannot see become measurable, repeatable, and comparable across models.

The point is not that static benchmarks are wrong. They measure what they measure. The question most people care about is whether a system is a good advisor when it matters. A static test cannot answer it. It is an experiment, and it has to be run as one.