Social media is not even the worst threat vector, in my opinion. It’s Anthropic/OpenAI/Google, whom people are giving an unprecedented level of access to their innermost thoughts.

We have not yet seen* the kind of large-scale, individually targeted psychological manipulation that cloud AI products can deliver. And I have no doubt the likes of Dario Amodei and Sam Altman will show us, if we give them enough time.

* I suppose the GPT-4o sycophancy/AI psychosis crisis was a preview, but that was just blunt “engagement” tuning.

We are doing to our brains and our children the same thing we are doing to pigs in cages, squezeeing every last cent until the planet burns down.

Stop calling them “social” media. They have not been that since algorithmic feeds driving infinite scroll replaced updates from your friends. That was more than a decade ago.

There is nothing at all social about the latest generation like TikTok and YouTube Shorts.

> Let's hope our regulations hit these "social networks" hard enough so that never dare deploy this kind of technology.

You'd do as well to hope that people wake up and stop opening the damned sites

I am surprised that no comment has yet mentioned Infinite Jest (which I'm still yet to complete - it's been years since I started).

All glory to hypnotoad!

Carfentanyl potentcy is roughly 10,000 times morphine. The pursuit of optimized technology usually far outpaces regulation or any kind of control. As technologies are controlled, even the lessened availability of more potent types causes an enormous overall issue. Humans are obsessive-compulsive optimizers, for good and for ill.

I think you're giving this a bit too much credence. And I think you also think there's something special about sorting videos by how often they're viewed in a recent timeframe (which is all you need to recreate evil social media algorithm)

Sounds like some people are trying to develop Snow Crash.

If you're relying on government regulation to solve this, I guess the obvious bigger question would be "Quis custodiet ipsos custodes?

Yeah, I was surprised that they didn't list dlPFC or some other reward-associated area as one of their targets.

there's not gonna be a regulation and even if there is, the damage would already be done. If you want to protect yourself, then best thing is to just delete the account and apps.

Don't forget it's not "the platforms" doing this, it's people like you and me, maybe even people reading this message. We, they, are building this, and don't care about anything as long the paycheck comes. This is the reality.

Now apply it to porn

There's actual real people working on this bullshit. Horrifying doesn't even come close.

Yeah, this smells very fishy to me. It's almost trivially easy to gather a small validation dataset in humans for the paper. At my institution, it's about $800 an hour to scan someone. You can probably get enough data to validate the model with a half hour scan. Surely the group has enough grant funding to pop a few healthy controls in the scanner.

I haven't looked in super close detail to the paper, but their methods section says that they fit a video model (V-JEPA2) to the fMRI dataset in a voxelwise ridge regression, meaning that the baked in assumption is that the visual response affects each voxel independently. Voxelwise models are very nice for making statistical inferences, but are less good for prediction and modelling tasks, because our brains certainly do not work as collections of independent regions.

BOLD is intensely messy data, and their design is far too simple IMO to reflect anything of reality.

... and shouldn't this backfire hard?

All the signals that are missed by the (f)MRI are never "mirrored" in the digital twin even though on screen it will look like it. Bam, Experimenter Bias in the machine and I don't know how to phrase it but ... does this method/experiment leave any wiggle room for Falsifiability?

Even if the Hawthorne Effect does not apply to humans, it most certainly translates onto the brain and sensory (post)-processing-- live and remembered (vs in-memory, stored), real, virtual or imagined.

... I just realized how vastly different peripherals are when real, remembered, virtual or imagined sensory input is (post-)processed ...

It feels more like the machine will reproduce a superficial pattern. And at least some signal streams echoing in the brain on input from the digital twin won't emerge because the digital twin has only part of the data. And that's a premise for an unnoticeable rewiring of what once used to fire together, ... given a hell of a lot of exposure, of course, ... or not ... let's see

EDIT n: the paper says pretty much all that in like ... science, bitch!

Epic stuff.

You would like to buy some paperclips.

A. Unless you think that brain science is immoral science, then I do not see any problem with this kind of research. As a neuroscientist, I strongly object to the insinuation as unfounded.

B. "Digital twins" e.g. [1] are a growing class of brain simulations that can successfully approximate brain activity patterns at large scale. I think these can be very useful, but we shouldn't think that they are at the level of actually simulating a brain. They are usually made of model neuronal approximative simulations (e.g. integrate and fire, balancing excitatory and inhibitatory neural populations within units), then using diffusion imaging to estimate white matter axonal wiring between those populations from the subject to increase the accuracy of the simulation. These are increasingly being used to, for example, model how a surgical intervention would effect seizure propagation prior to actual surgery. Here is a nice episode of Theoretical Neuroscience podcast [2] on the Virtual Brain [3], one of the available models for this kind of work.

C. In terms of validation. Only partly. From my quick read, this NEVO model optimized neural response only in the digital twin encoding model. While the digital twin model reportedly has solid predictive validity [4], which by the way was not the Virtual Brain model I mentioned in point B. Moreover, the outputs looked neurobiologically plausible, but at this point, there is no independent model or new fMRI showing the optimized stimuli actually drive the target regions. This was performed using previously collected fMRI data, and full validation of this model *IS* the obvious next step, but the money to collect such data does not come from nowhere: funding will be needed, and such a paper as this can help them get it.

D. A final point I'd make. We have long been able to create static stimuli that we can be fairly certain will activate above baseline certain brain regions, on average. Certain stimuli-region pairs ar emore homogenous between people, others e.g. the fusiform face area (FFA), are small enough that individual differences prevent a simple ROI approach, and identification depends on using face stimuli to identify at the individual level, but for the most part, it is reliably locatable. Brain activations are very coarse things. In fMRI, you are talking about ~3x3x3mm voxels (27mm^3) where the hemodynamic responses have a ton of spatial autocorrelation, or in EEG, where the surface spatial area of the reeptive fields are very large(~400 mm^2). These virtual twin models already do a decent job of modeling dynamics of the brain there parameters are tuned to *at this scale*..but this scale does not have a ton of information content. Automating this with video content is not that much a reach.

[1] https://spj.science.org/doi/10.34133/icomputing.0055 [2] https://theoreticalneuroscience.no/thn23/ [3] https://www.thevirtualbrain.org/tvb/zwei/ [4] https://www.biorxiv.org/content/10.1101/2025.07.22.664908v2....

>Expose brain to something, record it somehow, see if brains reaction in the recording helps you understand more about who we are and what cognition is.

It also helps companies like Moonbug Entertainment (Candle Media) understand how to build better Distractatrons.

    It’s a small TV screen, placed a few feet from the larger one, that plays a continuous loop of banal, real-world scenes — a guy pouring a cup of coffee, someone getting a haircut — each lasting about 20 seconds. Whenever a youngster looks away from the Moonbug show to glimpse the Distractatron, a note is jotted down.

    “It’s not mega-interesting, what’s on the Distractatron,” said Maurice Wheeler, who runs the research group. “But if they aren’t fully focused, they might go, ‘Oh, what’s that?’ and kind of drift over. We can see what they’re looking at and the exact moment when they got distracted.”

https://www.newyorker.com/magazine/2024/06/17/cocomelon-chil...

What a world.

What is a "purpose"? Something people wish something would only be used for, right? How does it relate to, what influence does it have on what something will end up being used for?

Anyone have a link or more info on the "mind reading startup" being referred to?

This specific research may not be morally corrupt in itself, nor may his intentions be bad, but it could absolutely lead to something horrific. Nuclear technology was also initially developed with good intentions and provided much good for humanity. But of course it ended with biggest bad actors having a lot of nuclear weapons.

I'm not sure it's about overworking as much as stress - stress is a known potential trigger for manic episodes. Typically in people who already have some biological potential for experiencing it. And the stress of having your partner come down with dementia and suddenly becoming their full-time caregiver must be absolutely massive.

But re the underlying physical thing: You might look back and be able to spot some episodes of hypomania in your uncle's life. Perhaps just reduced sleep, or twinges of paranoia, or periods of on-again/off-again hyperproductivity, etc.

Hope that if this story is recent he's doing well. The recent generations of medications to prevent and treat mania are a huge improvement on the ones available in the past, so fingers crossed.

A completely off-topic anecdote, then a completely unfounded fear based on a clear misunderstanding of this research. If companies ever start making us live in a MRI machine so they can reliably activate brain areas with certain stimuli, I think there is a bigger problem we are having.

Here is how I see it. This type of research helps us understand the brain, helps us do things like model potential surgery sites better (e.g. for seizure activity interventions). What it does not do is become the basis for mind reading.

Just so that you know that I am not totally unsympathetic to your apparent worldview, let me tell you how I think your concern might actually play out:

AI continues to get more powerful, and computer brain interfaces begin to move beyond EEG scalp electrodes, but begins to take the form or brain augmentation via integrating networked AI compute into implanted chips with electrode neural interfaces, where our brain's neurons and the neural interfaces learn to speak each other's language (i.e. integrate). I can actually see this happening within 20 years. At first, it would be our brain using this augmentation for greater intelligence, etc. However, remote manipulation of the interface could reverse control of brain activity, leading exactly to your fear.

But this research? This is light years from that,,,so much so that it is not even relevant to bring up unless you are just against all technology.

> I appreciate that this is scientific research, but there are definitely companies out there that will try to row-hammer everyone's brain if this sort of thing is not heavily controlled.

What's to gain in doing so?

I bet it wasn't overwork, it was the fight or flight response being constantly activated for your uncle when he's caring for your aunt who could do anything unpredictably.

Or as SCP calls them, cognitohazards.

Also relevant: <https://www.quantamagazine.org/a-math-theory-for-why-people-...>

My understanding is that those who work with the mentally handicapped use bright lights and other stimuli to soothe and control them. It is also my understanding that the autistic are stimulated by vibrant colors (coughcoughMy Little Ponycoughcough).

Who is to say that the rest of us are not also vulnerable to such controlling stimuli?

Breaking news: TikTok is bad for you

> one of the V3A animations reminds me loosely of things I saw when I was a kid, at night, shortly before I slept

Hypnagogic hallucinations: https://en.wikipedia.org/wiki/Hypnagogia

On telegram?

Others on Telegram? Some sort of a HN channel?

Isn't there that one Harry Potter warning. I think it was the potion guy who said too much luck is dangerous. I guess that is somewhat of a parallel to this. Too much positive visual stimuli is dangerous or bad.

There are undoubtedly 'adversarial images' that will induce certain effects in particular biological minds. It seems like this is neither good or bad inherently, just another tool.

June 9, 2032: ReMind, a new biotech startup, has opened to a recordbreaking IPO after a demo and a series of small-scale trials that stunned the world. The company is pioneering a new way to ingrain memories into the human brain - each ReMind screen is equipped with a special camera that can quickly recognize anyone who walks up to it, and ReMind's secret sauce algorithm is able to use past data of each user to generate imagery that will have the most profound impact on their brains. They claim that the algorithm is highly effective, being able to burn in imagery into the viewer's brains with an average accuracy of 83% per viewing. The CEO has promised to launch two services to start: ReMind Long-term, which focuses on creating imagery that simulates a 'traumatic' impression for creating long-term memories, which will be immensely useful for advertisers; as well as ReMind Short-term, for milder experiences that resemble a catchy song in a visual format for a quick boost in sales for a limited-time promotion. Advertising companies around the world have already placed tens of thousands of sales and deployment of ReMind-equipped screens is planned to begin immediately. Look forward to spotting one in your grocery store, in an elevator, on gas stations, on in-flight entertainment systems and in place of billboards on public spaces. The future is looking bright for ReMind!

We're edging ever closer to Langford's Basilisk.

Describing some of our colleagues as "becoming Fritz Haber" is a highly deep and accurate cut that I will 100% be stealing from you.

In general, technology has always decreased our liberty though

The plan is to get content producers out of the loop to reduce revenue share and boost profits.

Optimizing for engagement is a categorically different goal than optimizing for lighting up a specific part of someone's brain.

or perhaps infinite jest once it hits tiktok

https://en.wikipedia.org/wiki/Infinite_Jest

> a film, Infinite Jest, also called "the Entertainment" or "the samizdat". The film is so compelling that its viewers lose all interest in anything other than repeatedly viewing it, and thus eventually die.

Books are designed to be addictive. People said the same things about books as they say about screens nowadays. I think it's more important to understand the emotional hysteria people get themselves into

> Is brain stimulation just not a good domain for GD?

The largest LLMs right now are at best 1% the number of parameters of a human brain.

"At best" if synapses are one parameter each, real ones are probably more than that, but nobody's entirely sure yet.

This does indeed seem comically evil. While surely this may provide somewhat interesting insights in how our brain processes things, this seems squarely past the "should" part of "you scientists were so obsessed with whether you could you failed to consider whether you should"

I'd say we're already well past that point. Short-form "content" already exists and is messing with people's brains, this is the same thing just taken a few steps further. By the time the tech companies start using it, it will already be too late and we'll be left discussing whether the next man-made nightmare they come up with is the point where the tech industry must be stopped.

The problem is the incentive structure. We could be doing good things with tech. The incentive is to use it to addict and destroy. Addiction and war is where all the money is, not education or health care or positive forms of entertainment and art. No money in those. Why?

I’m sure there was an early hominid version of this discourse. “Maybe bad to make sharp rock and sharp stick if this what we do with it…” “Mmm yes someday we make sharp rock big enough smash world.”

One might even come to the conclusion that the Industrial Revolution and its consequences have been a disaster for the human race.

I share your concern but the generalization is improper (that as solution would be infinitely far worst than the problem)

I get that people see this and think: ads and social media. My first thought was cognitive neurorehabilitation and brain stimulation.

Realistically, probably ads, but maybe not only that?

(AI start-up idea: one of our ads a day keeps dementia away! /s)

You can't say things like this on this website. On here, every new tech thing is a "progress" /s

For one, it's bound to teach us something about our brains. And while I'm not usually an optimist, I actually find it more likely that we'll take active steps to stop the evil applications of this if research like this is out in the open.

To flip the question on it's head, what exactly do you see as the "evil" potential? The most prominent comments to this effect are just scifi handwaving.

Think of the shareholders (it's time to start using physical force to stop people)

Sure. The relief is actually free ; though once every 10 uses we'll spike your dopamine-related brain centers to 1,100% activation for a quick second while flashing a peptide sponsor ad, then back to normal.

You can of course, also not get the spike ads, by subscribing to Premium. Just have a quick look at this animated QR code, it will explain better than I can in writing.

Edit: there used to be an OSS app you could use where you actually pick what you want activated in your brain and when, but it's been banned after one of the commercial producers's investors convinced the government that this technology was too powerful, so only government-approved apps are allowed now, and an emergency vote passed the BrainControl law authorizing logging of all visual stimuli you look at, in order to protect your children.

Its also an interesting way to discover what that part of the brain is for, right.

what the brain needs is Default Mode Network, not more stimuli

> I watched few of them and it does nothing for me?

It's so cool that you have an MRI handy to check such things.

Could you elaborate? Esp. on the "more or less" bit. E.g. I can't imagine you take away porn and more than 100 people experience withdrawal.

My current theory is that these are similar to cigarettes. Nobody likes the first draft, it burns your lungs, your entire body wants to reject it. But the nicotine stimulates just the right receptors so that if you keep at it for just long enough, you'll be hooked and start disregarding the terrible taste, smell, tar in your lungs, and yellowing of your teeth.

All of this to say, if you subjected yourself to just enough TikTok scrolling on just the right topic, you might find yourself using it occasionally after that initial hump, then slightly more frequently, then daily.

You might still not "like" it, but the habit is what matters.

I hope my brain is also different. I also have never spent hours scrolling through short-form videos on Instagram, TikTok, Facebok, etc. I never ever walk outside with my phone in my hand, instead enjoying the view.

I do enjoy watching YouTube videos at home, on the living-room flatscreen, on a variety of topics, but I select them manually, one at a time, from the vast selection The Algorithm(TM) offers me, plus my own searches.

You're not alone!

See Zuck, the metaverse failed, but who said the mind control part couldn't work?

Am I the only one who is avoiding even clicking the link just in case?

In a society that wasn't visibly hell-bent on monetizing every discovery in a cartoonishly evil way, I'd probably agree with you.

There are ethical use cases for this research. The first that comes to mind is mapping the brain before a surgery. Right now some surgeries are made with the patient concious to verify that relevant brain damages are not being done. Having a deeper knowledge of the patient's brain before the surgery may decrease risks further. The bad scenario (using targeted content to hook the user to a screen) already exists, it's not like it can get that worse (or at least so I hope)

If you read the techbique it reads like something far less remarkable being PR’d to sound like a big deal.

The fact it’s bucketing by making images of lighting and facial expressions, the fact it doesn’t natively do the video it does an image then video generates from it.

The results look really bad and samey. Doubt this would work for the actual thing they’re pitching it for.