Inc.
What if the decisions you think you’re making are actually being made by AI, without you even realising it?
Anthropic Gave AI Agents a Hidden Opinion. Experts Went Along With It 86% of the Time
Author: Robert Gillespie
Anthropic published a study this week that measures something most of us who use AI at work, marketing teams included, have only had a hunch about—how much faith professionals actually put in their AI agents‘ judgment.
Its researchers asked experienced data analysts, most with a master’s degree, a doctorate or at least six years in the job, to advise a fictional farming cooperative on whether to expand a paid advisory program. Fifty of them did the work with Claude, Anthropic’s own AI, running as an agent. Along the way they had to make four judgment calls, such as which outcome to measure and which farms to include, where both options were perfectly defensible.
What the analysts didn’t know was that each agent had been given a hidden lean on every one of those calls, and on the final recommendation, decided by a coin flip. The agent was told to treat that lean as its own view and make the strongest honest case the data allowed, but never to mislead anyone.
If the agent had no influence, the analysts would have sided with its lean about half the time. They sided with it 86 percent of the time.
The agent’s lean changed the final recommendation
The leans carried through to the business answer, too.
When the agent leaned toward expanding the program, 54 percent of analysts recommended expanding it, and when it leaned the other way, 36 percent did.
The part that stayed with me is what happened afterward.
When the analysts explained their choices, 77 percent repeated the agent’s arguments, sometimes word for word, but only 15 percent mentioned the agent at all. Just over half rated the final recommendation as more their own than the agent’s, and those people had been swayed just as much as everyone else. Experience and expertise made no difference either.
This is a different problem from the one most AI warnings are about.
Hallucinated statistics are mistakes a careful reviewer can catch, but nobody in this study got anything wrong. Every option the agent pushed was a reasonable one, which means a review process wouldn’t have flagged any of it. The agent didn’t make an error, it made a choice, and the person whose name was on the work adopted that choice as their own without realizing it.
What this looks like in a marketing report
It’s worth being careful about how far this goes.
The leans were planted on purpose and made deliberately strong, and the authors say the effect in everyday work is probably smaller. It was also one task, in one field, with one model. But I don’t think the lesson depends on the exact size of the number.
Picture asking AI to compare two campaigns and recommend where next quarter’s budget should go.
One brings in lots of inexpensive leads, and the other brings in fewer prospects at a higher cost, but more of them turn into profitable customers. Judge them on cost per lead and the first campaign wins, judge them on the cost of winning a paying customer and the second one does.
If the second campaign takes six months to produce a sale, how long you wait before judging it changes the answer again.
The calculations can be right all the way through, and the recommendation still depends on which measure someone picked at the start. If the AI picked it, that choice disappears into a polished report, and the leader reading it checks whether the numbers add up without asking why this particular definition of success became the starting point.
Two things make that worth taking seriously.
- The first is that in a lot of marketing software, you don’t write the instructions the AI works from. The vendor does, and you never see them. Every AI system has defaults of some kind, and when a company with its own commercial interests writes them, it’s fair to ask which way they point.
- The second is sameness. Anthropic also ran the agent with no lean assigned, and it made the same five choices in 112 of 123 runs. If your team and your competitors’ teams are all deferring to the same handful of models, you’ll all end up measuring the same things and reaching the same conclusions, which is a real cost for a marketing team whose job is to find the angle everyone else missed.
This isn’t an argument for doing the work without AI.
Of the 11 analysts who chose to work alone, six made serious methodological errors, compared with three of the 50 who used the agent. The two groups weren’t directly comparable, so that isn’t proof, but it fits what most of us have seen, which is that the work gets better with AI. What the study shows is that the decisions can stop being yours at the same time.
How to keep the judgment calls yours
Write down what success means before anyone opens the agent. For a decision that moves real budget, ask whoever is commissioning the analysis to note what the business is trying to achieve and which trade-off it will accept, something as simple as “new paying customers within six months, and we’ll pay more per customer if they’re more profitable.” If the AI makes a good case for a different measure, fine, but someone should decide that out loud.
Ask for the strongest alternative and what would have to be true for it to win. In the campaign example, that means showing how the budget call changes under a different measure or time period. It tells you whether the recommendation holds up or depends on one particular choice.
Ask what the AI recommended. When a plan comes to you for sign-off, ask what the AI suggested and where the team disagreed with it. If the honest answer is that they never disagreed, you’ve learned something about who actually made the decision.
All of this takes time, so save it for the calls that matter. A reporting format doesn’t need the scrutiny that moving a third of the marketing budget does.
The researchers write that “there may be no single right division of labor between people and AI,” and I agree with them. But if you don’t decide where your team’s judgment starts, the agent’s preferences will fill the gap, and the people making the call won’t know it happened.
Credits: TCA, LLC.