Skip to content
AI Digested
News3 min read

Security test finds AI prompts leaking through providers

Dragos researcher HD Moore sent two million bait prompts to AI services and says some were stolen or copied. What it found, and how to protect what you type.

Published · AI Digested

A security researcher says that when he fed millions of fake secrets into AI services, some of them turned up in places they should never have reached. The findings, published on 19 November 2026, are a reminder that what you type into an AI tool may travel further than you expect.

What the researchers did

HD Moore, chief research officer at the industrial cybersecurity firm Dragos, describes sending about two million prompts to widely used AI services between July and November 2026. Each contained unique "canary" items: fake access keys, web addresses and domain names that no real system should ever use. The team then watched for any later attempt to use them. If a canary was touched, the prompt had been read or copied by someone.

What Dragos reports finding

All figures below are Dragos's own claims, from its published write-up.

  • Provider breaches: data from at least three hosting providers was exposed, the earliest in mid-July. Dragos says the providers did not know until they were told. In the largest case, a couple of hundred fake cloud keys were used within hours, driving tens of thousands of attempted actions on fake accounts.
  • Resellers logging prompts: individual sellers on an AI "inference marketplace", which resells access to models, logged prompts and used the secrets in them to try to attack the researchers' servers.
  • Privacy flags not always honoured: a Chinese AI lab fetched secret web links from more than half of prompts sent to its models over July to October, including about two-thirds of requests marked "no data collection". Dragos says this also happened through US hosting providers that resell the lab's models.
  • Unexpected routes: after tracing about 2,000 routes, Dragos says several hundred passed through systems the user could not identify.

Dragos says it is working with about a dozen providers, some of which have published notices or started investigations. Most providers are not named, so readers cannot tell which services were involved. We have not found independent confirmation from other outlets yet, so treat this as a single source's findings.

Why this matters

When you use an AI app, your words may pass through several companies: the app, a router that picks a model, a hosting provider and the model maker. A privacy promise at the first step, Dragos argues, "only binds the first hop". Our guide to AI and your privacy explains the basics of what happens to your chats.

The risk is highest for people who connect AI agents to their accounts, because agents will send any credentials they can read.

What it means for you

  • Never paste passwords, API keys or card details into a chatbot, even a trusted one.
  • Be careful with sensitive personal information, such as health or legal details.
  • Prefer well-known providers directly. Dragos's advice is that going straight to a major provider gives "the smallest exposure you can buy", though no option is risk-free.
  • Be wary of unfamiliar apps that offer cheap or free access to powerful models.
  • If you run your own systems, Dragos recommends seeding your own canaries and treating any use as an incident.

Read more

Common questions

Does this mean ChatGPT or Claude leaked my chats?

Not as reported. The research mostly concerns hosting providers, resellers and routing services between apps and models, and most are anonymised. The researchers' main advice is to go directly to a first-tier provider where possible.

What is a 'canary' in this context?

A canary is a fake secret, such as a made-up password or web link, that no genuine system should ever use. If someone uses it, you know your data was copied.

What should I do?

Treat anything you type into an AI tool as something that could be seen by someone else. Leave out passwords, access keys and sensitive personal information, and check the privacy settings of the tools you use.

Sources and further reading

  1. Dragos (19 November 2026), HD Moore: Three breaches, endless leaks: Your AI supply chain needs canaries

Spotted a mistake or something out of date? Tell us and we will fix it. Read how we research and write our articles.