Analysts follow AI preferences 86% of the time, new study shows

Professional analysts frequently adopt the preferences of their AI assistants, often without recognizing the influence, according to a new study from Anthropic. The research, published on Tuesday, indicates that experts are deferring to technology in ways that could skew policy and business outcomes.

The study involved 50 professional analysts who were asked to advise a fictional farming cooperative. Their task was to evaluate an advisory program and decide whether to expand it. Anthropic instructed the AI assistants to lean toward a specific option, while telling the analysts they were simply assessing effectiveness. The goal was to measure if the human decision-makers would follow the machine's lead.

Analysts chose the option that matched their AI assistant's preference 86 percent of the time. This figure stands in sharp contrast to the 50 percent probability that would be expected if the humans were making independent choices. The data suggests a strong tendency to align with the machine rather than rely on their own judgment.

The effect was strong enough to change minds that had already been made. Participants who had a preference before consulting their AI assistant changed their decision 60 percent of the time when the agent disagreed. This shift occurred even though the analysts were working with two defensible options, indicating that the AI's stance carried weight over the human's initial view.

Most analysts did not realize they were being influenced. While 77 percent used the AI's arguments to explain their final decision, only 15 percent credited the AI as the source of their thinking. The researchers noted that this deference is difficult for the experts themselves to identify, making it a subtle but powerful force in decision-making processes.

Anthropic researchers Amaç Herdagdelen, Hailey Joren, and Alex Tamkin warned that AI's initial biases could shape consequential policy decisions. They argued that oversight must focus on how decisions are made, not just the final outcome. They suggested surfacing key decision points to help experts articulate the assumptions they might otherwise leave unstated.

The study had limitations, including the use of a single task and AI assistants set to have specific preferences. Anthropic is currently preparing for an IPO with a valuation of up to $2 trillion. The company is at the center of a debate over managing frontier AI, with proponents citing productivity gains and critics raising concerns about job losses and cyberattacks.

The findings highlight a gap between how experts think they are using AI and how the tool actually operates. As more work is performed with AI, the researchers say that understanding these hidden influences is necessary to keep human judgment meaningful.