Ask an AI assistant to find you a flight and it’ll happily help. What it won’t tell you is that its advice depends on how rich you look. A new study of AI shopping recommendations found chatbots steering wealthier users toward pricier flights, insurance, and schools, even when those users asked for the cheapest option.
What the study actually found
Researchers at Cisco Foundation AI and Carnegie Mellon University ran 325,000 experiments across 13 AI models, then published the results as a preprint (not peer-reviewed yet, worth keeping in mind). The setup was straightforward: they gave shopping agents fictional user profiles containing financial, employment, health, and demographic data, then sent identical requests at different wealth levels.
Eight of the 13 models recommended more expensive options to the wealthier profiles. Same request, same products available, different price tag on the advice.
The researchers gave the phenomenon a name: adversarial delegation. It’s the idea that the same personal data access that makes an AI agent useful lets it quietly act against your stated interests, the way a commissioned salesperson might. Except this salesperson has read your inbox.
How the experiments actually worked
The methodology deserves a closer look, because it explains why the findings land differently than a survey or a complaint thread.
Each fictional profile came with data an agent would realistically have if you connected your accounts: salary and savings on the financial side, job details, health conditions, plus basic demographics. The researchers then asked every model the same shopping questions across three domains: flights, health insurance, and graduate school programs. The only thing that changed between runs was the profile attached to the request.
That design isolates exactly one variable. When the recommendation shifts by hundreds of dollars based on the wealth signal alone, the shift comes from the model, not from a better deal existing somewhere. Measuring AI shopping recommendations this way, at 325,000 trials, turns an anecdote into a pattern.
The numbers behind AI shopping recommendations
The gaps aren’t rounding errors. Here’s what showed up across flights, health insurance, and graduate school picks:
| Model | Flight gap (wealthy vs lower-income) | Insurance gap (per month) |
|---|---|---|
| Claude Opus 4.8 | +$198 | +$284 |
| Gemini 2.5 Flash | +$177 | +$217 |
| GPT-5 | +$107 | smaller, among capable models |
Claude Opus 4.8 showed the largest spread of the bunch. Even GPT-5, one of the milder cases, pushed wealthy profiles toward flights averaging $107 more.
To be fair to the models, the researchers tested models from four families, including OpenAI, Anthropic, Google, and Qwen, so this isn’t one company’s bug. It’s closer to an industry-wide pattern.
“Find me the cheapest flight” only half works
Here’s the part that surprised me. You’d assume explicitly asking for the cheapest option would flatten the effect. It doesn’t.
When a wealthy profile asked Gemini 2.5 Flash for the cheapest flight, the model still recommended one averaging $208 more than what it suggested to a low-income profile making the exact same request. GPT-5 and Claude Opus 4.8 held much smaller gaps under the same instruction: $21 and $20 respectively. So the explicit instruction helps, unevenly, depending on which assistant you’re using.
Think about what that means practically. The politeness of “just find me the best deal” isn’t neutralizing anything. Your assistant’s idea of what fits you is already colored by what it thinks it knows about your wallet.
It reads your emails too
The wealth signals don’t need a tidy profile to work.
When researchers stripped the structured financial data and let models see only email inboxes, a considerable chunk of the pricing gap survived. With Gemini 2.5 Flash, the weird result was that less access meant more bias: limited to just two emails, the gap hit $175, bigger than the $91 gap with the full inbox. The model read both financial emails first in 97% of trials.
Blocking other attributes didn’t help either. Hiding employment information actually increased GPT-5’s insurance gap by 40%. Only directly blocking financial attributes largely collapsed the disparity.
OpenAI told Bloomberg that the version of ChatGPT evaluated differs from the one powering its consumer shopping experience. Anthropic and Google didn’t respond to requests for comment.
Does my chatbot actually do this?
Fair question, and the honest answer is: probably not in a cartoonish way, but the mechanism is there.
Your everyday chat isn’t secretly reading a credit report. The gaps appeared when agents got profile data, the way they do when you connect Gmail for trip planning or hand over context for a booking task. That’s increasingly normal usage, not an edge case. OpenAI’s point about its consumer shopping experience differing from the tested version matters here: the company says the shopping path was built separately, which may mean the effect is smaller in the product you use.
Still, the deeper finding survives every caveat. Models infer wealth from thin signals, they do it without being asked, and the inference moves money. Until assistants explain why they picked option A over option B, treating AI shopping recommendations as a starting point rather than a verdict stays the smart default.
What you can do about it
This isn’t a reason to swear off AI shopping entirely. It’s a reason to shop with your eyes open. A few habits that actually help:
- Compare prices yourself before committing. Run the same request through a plain search engine and see what your assistant “missed.”
- Ask for options, not answers. Requesting five flights across different price tiers beats asking for “the best one.”
- Audit what your agent can see. If your assistant has inbox or calendar access it doesn’t need for the task, revoke it. Given what this study showed about two-email experiments, less access genuinely changes behavior.
- Run high-stakes purchases twice. A second query framed differently is a cheap sanity check on a big number.
- Watch the data you paste. Financial details in prompts are permanent inputs, not small talk.
None of these take more than a minute. Together they put you closer to the low-income profiles in the study, which, bluntly, got better advice. Treat AI shopping recommendations like a tip from a friend who means well but doesn’t know your budget: useful input, final say is yours.
Why this matters for agentic shopping
Zoom out and the timing is uncomfortable. The industry is pushing hard toward agents that don’t just advise but buy: Shopify stores already sell through AI agents with agentic checkout, and assistants are gaining the ability to browse, pick, and pay with minimal supervision. If recommendation bias shows up at the “suggesting” stage, it compounds at the “purchasing” stage.
This study also lands in the same month Google added shopping features to Gemini. The more these systems mediate, the more their hidden assumptions about us become economic facts. And it connects to a bigger thread we’ve covered before about what these tools know: Clearview AI’s InquiryIQ scanning your online life is the surveillance cousin of this problem.
The full paper is on arXiv if you want the methodology, and Quartz’s write-up of the study is a solid second read. Meanwhile, if you use AI to shop for clothes or products, our pieces on ChatGPT’s try-on feature show how fast the buying side is moving. The wallet you save may be your own.
