Tangential:

I have recently noticed that e.g. ChatGPT, when used from a web browser, periodically sends unfinished prompts to their servers, namely to the `conversation/prepare` endpoint, without waiting for the user to actually send it.

This partial prompt data might potentially be used to "pre-warm" some kind of cache.

But it may also be used to track the user's writing cadence, error correction style and evolution of their stub ideas as they are being formulated into a prompt. I would assume that such data could also be sold to the advertisers.

It's the same lesson as the Navier-Stokes credit fight earlier this month. Buckmaster and Alpoge had their unpublished drafts in private Codex sessions and OpenAI says nobody saw them but admits de-identified product data may have improved its models. There it's training data, here it's ad trackers. Either way, prompts and results that should stay private don't. Thats why even though open models aren't perfect it has to win. You can skip the app and run the model yourself.

My least favorite trend I’ve noticed with so many AI chat services is they seem to equate a UUID in the url with privacy.

Perplexity does this. Visiting a past perplexity search url exposes your full conversation.

In an old Simpsons episode Lisa gets to visit the Teachers room, where all the staff are making fun of the children. Groundskeeper Willie is pantomiming Milhouse “Oh I am Milhouse, I tell all my secrets to Willie since I have no friends!” and the teachers laugh. Later something embarrassing happens to Milhouse and he immediately runs away crying “I have to tell this to Willie!”.

We have all become Milhouse now.

This is a bit surprising to me, considering how much AI companies love to hoard data. Especially since some of these ad companies are direct competitors!

My best guess is that these ad mechanisms are a bit rushed and/or that investor demands for profitability are fighting against company self-interest.

Edit: I guess some data will always need to be leaked for AI chat ads to be most effective. But I imagine AI companies would rather deliver the targeted ads themselves rather than letting competitors do it for them. It would be scary to see AI companies become ad companies too (instead of just hosting them).

A lot of people seem to be very in denial about the fact that OpenAI and co do not give a crap about you. They don't care about the agreements you've signed. You're just a pile of cash to them

Prior to this new AI age, your data was calculated and what was inferred about you was "shallow", but as of today. The sort of profile and things that can be known about you is scary especially if you are constantly engaged with cloud AI. IMO, the number one risk of using cloud AI is loss of privacy and loss of freedom. With AI and capabilities, more controls can be placed on people and the more you put yourself out there, the more you are going to lose.

For example, we now have self driving cars, we have cameras everywhere. Based on your chat with a cloud AI, you can automatically trigger an automatic monitoring event that follows and tracks you in the real world with the fleet of cameras, cars, GPU, cell signal. Your tracking due to AI has moved into the real world and eventually, a self driving car will take you in to be "processed" against your will, not even for what you posted in a public forum, but for your private ribbing and chatting with some cloud AI.

So definitely put local AI into the mix and keep personal stuff and thoughts local only.

The word is "sell" not "leak". This title takes away all agency from the thieves selling private data to advertisers.

Ad-tech spent 20 years trying to infer intent from clickstreams. Chat apps now hand over an AI-written one-line summary of intent, labeled and keyed to a cookie. Are we sure the ads business model in AI is about ads in the chat, and not the chat as the targeting signal?

Is it still called a leak if it was the whole point and purpose of the deal?

Someone should have to investigate, but I suppose it's all "legal"?

This paper focuses on the use of the providers' web and phone tools, and the data sharing arrangements they built through ad networks and tracking services like Data Dog. It doesn't talk about how they handle data from API calls. For one thing that can be pretty opaque.

I have a friend/client who understandably doesn't trust the existing privacy policies of the major providers. They have the same problem many of us do: We want the most powerful models, we're willing to pay for them, but we see over and over how much of a frontier the frontier actually is. Frontiers are ugly if you don't have guns.

So for now maybe platform tools like Open WebUI and TypingMind are a good workaround since the big boys don't (apparently) train on API data (for now) or (probably) send that data to advertisers. It would be interesting to confirm that.

In case anyone finds it helpful, I asked ChatGPT to break down what they found by app:

https://chatgpt.com/s/t_6abbbee386bc8191a26717b4f1442657

time for somebody to make a quick script to poison your chat history by starting 1000 fake conversations with contradictory identifying details "I worry as a lesbian woman, my tween daughter doesn't blah blah Shabbat blah blah move home to Australia"

It’s not a leak if it’s the business model.

I'd be kind of surprised if OpenAI were really doing this deliberately because a whole bunch of their execs come from Meta, and all those guys learned the hard way.

First: You don't want to leak information about your users to advertising networks because it's going to leak, get back to your customers, they're going to figure out you're doing it and get really angry.

But second and more importantly - it's a much better business model to collect that data for yourself, keep it in house and then you control how you use that data to target ads which gives you a massive competitive advantage in selling ads because you have unique targeting data.

The way meta does this now is the model, they don't give the advertiser a list of the people you're going to show the advert to, the advertiser gives you a list of characteristics they want to hit and meta decides who those people are.

Not surprising. When privacy is a selling point on the enterprise plan regular users, even paying ones, are the product. OpenAI had this from day one, the writing was on the wall. We all knew it so let's not act surprised now.

[1](https://chatgpt.com/pricing/?type=team)

"multiple providers disclose sensitive conversation-derived artifacts — including titles, prompts, and screenshots — to third parties, often alongside persistent user identifiers that enable user attribution. We also find that some providers publicly expose conversation permalinks without access controls, allowing trackers to read the entire conversation."

Not good at all.

Title is: Prompt like a Butterfly, sting like a tracker

or A Privacy Analysis of Web and Mobile Conversational AI Agents

Site link with context: https://jorgegarciaherrero.com/en/prompt-like-a-butterfly-st...

You should really assume that any information you give to a private model is going directly into a database. These companies don't care about your privacy. Local, open weight models are the only way to truly protect your data.

I would be curious if these concerns survive for chatgpt if you switch off all the marketing-looking stuff in their settings. They let you turn off a bunch of cookie types and there are a couple of useful looking toggles under "marketing privacy".

This is why I built my own chat interface. https://ai.ivx.run/

Access through: https://ai.ivx.run/chat/

You thought... they didn't?

the 'leak' framing lets them off easy, this is the ad business model working as designed. the part that actually stings is it's on the free consumer apps, which is where people paste the most personal stuff.

The web vs. mobile comparison is interesting. I’m curious how much of the privacy risk comes from the agent itself versus the platform APIs and permissions around it.

All sells but I saw Chinese models are upfront about that most of the time.

Ah, where are the good old days of you just paid for a service!

Why do you think they are fighting so hard to dethrone Google?

Google is an ad company.

The paper doesn't say when the app sends the conversion artifact.

I never thought this could happen (XD)

The ads ~~industry~~ racket is a cancer upon civilization.

This shit needs to be stamped down by law.

People take up pitchforks and torches against AI data centers,

well how much time, money, and resources have been wasted on advertisements over the centuries?

How many people, including children, have been deceived by ads?

How much privacy has been violated in the name of "sErViNg YoU rElEvAnT aDs"?

Have you ever tried browsing YouTube from a poor connection, and noticed how long it takes to serve ads? How much bandwidth has been wasted on ads so far?

Who[m] does it all benefit?

I am very surprised.

Hanlon's Razor given that these tools are almost certainly vibe coded at this point?

my first guess is always Gboard.

Yea no shit sherlock!

My stance used to be that the invention of the filter bubble combined with targeted advertising is the most dangerous invention for human society of the last 100 years.

AI turns that up to 11

data is sold i think

Not surprised. Just wait until they bake the goddamn advertising right into the model.

Hard to read on mobile. Is there a tldr of how bad each provider is?

Oh no, what, what happened?

What happened? Oh no!

How terrible! That’s just, that’s just awful!

How terrible! Oh no!

Insert surprised pikachu meme

these companies are complete dumpster fires and we just keep giving them more to burn

Entire conversations via exposed permalinks. For Grok: trackers receiving the conversation URL could access the full chat because the link lacked access controls.

Screenshots of conversations. TikTok received screenshots of Grok chats during sharing, exposing the actual visible conversation content.

Conversation-derived content tied to persistent identifiers, including prompts and automatically generated chat titles revealing sensitive facts. "Salary 85k NYC: mortgage 280–350k".

This is exactly the kind of rigorous analysis the AI industry needs. From our hands-on testing of 500+ AI tools, we've seen the same pattern: most conversational agents ship with telemetry baked in, and very few disclose what's actually being collected beyond the standard "we may use your data to improve our services" boilerplate.

The prompt-injection angle is particularly concerning — it's not just about passive tracking, it's about the agent's context window becoming an attack surface. When a customer support agent pulls in untrusted web content or email threads, every loaded prompt becomes a potential data exfiltration channel.

More tools need to follow the local-first model (like tools that run entirely on-device or via self-hosted infrastructure) as the default for sensitive workflows.

One important limitation is that the paper excludes enterprise and government tiers, so I would not automatically map these findings onto a contracted business service. But it does suggest a better procurement test than asking only whether prompts train the model. For an approved workplace assistant, document the exact client and tier being used, inspect outbound domains from both web and mobile clients, test reject-cookie and no-sharing configurations, and ask the supplier to identify subprocessors, retention periods, conversation-link controls, and material change notifications. Repeat the test after major client updates. Where verification is not possible, restrict the data classes users may enter. I would also distinguish observed third-party contact from proof that full prompt content reached every endpoint.

[dead]

[flagged]

The shared links are bad, but persistent memory makes the blast radius much larger. At that point the product holds a durable record of what someone has worked out over time, not just one conversation. That needs real access controls and revocation, not a UUID.

[dead]

Of course they do. As soon as ads entered the equation this was always going to happen. Nobody should be suprised.

No, they don't "leak data", data is sold. Leaking data requires a mistake. This is intentional.

[dead]

[dead]

[dead]

[dead]