Partially agree. However, this problem has existed with scam e-mails since the 90s.

For me the solution is in signed e-mails and signed documents. If the person invites me to a online meeting with a signed e-mail, I trust that person that it's really them.

Same for footage of wars, etc. The journalist taking it basically signs the videos and verifies it's authenticity. It is AI generated, then we would loose trust in that person and wouldn't use their material anymore.

Laws will be passed to make it "safer". Just like it is happening with the id verification systems. Every image or video gen will require a watermark. Something visible which cannot be removed easily or hidden which can be detected and blocked. Access to models which do not comply will be made harder through id verification checks or something.

There will be some regulatory capture in between.

World will kick into gear only when something really bad happens. Maybe a influential person - rich or politician - fooled into doing something catastrophic due to a deepfake video/image. Until then normal people being affected isn't going to move the needle.

> Information found online will also no longer be trustable

Most information you can access publicly, including Wikipedia, is a result of astroturfing fight. Most information online had not been trustable for double digit number of years now.

> we already experience misleading articles today

Again, had been happening for decades.

> footage of some incident somewhere may have been entirely fabricated by AI

Not like we did not already have doctored footage plaguing the public.

> Money will have to be wasted on unnecessary flights to see stuff or meet people in-person instead of video

Necessity to inspect the supply chain for snake oil has been a thing since at least EA (the Nasir one).

We may be dealing with the problem of spam, but the problems have already been there.

We need some sort of end to end verification. Aka from the sender camera to the receiver display / speakers.

Maybe Apple will be able to pull it off? Aka if you FaceTime me I know that you are a person

"Is this a deepfake video call" is a major plot point in a pretty big movie currently in theaters, so I think this is getting into the broader zeitgeist.

How do you do when people don't protect their signatures? there is already scam where people get tricked into forwarding message from their own numbers to other people or email.

> footage of some incident somewhere may have been entirely fabricated by AI,

Or the opposite, where people attempt to get out of trouble by calling real evidence into question by calling it “AI”

We are still in the early stage of AI and already I struggle to tell what is real or fake on my Twitter feed. It will only get better in its deception with time.

You know those incriminating Epstein photos with his associates? A few years from now a common defense from people like that would be that the photos were AI generated, and it would be difficult to prove them wrong beyond reasonable doubt.

People in previous cases already attempted to dismiss incriminating pics of themselves as being the work of clever Photoshop artists.

What's the solution apart from an identity providing service?

Honestly? Maybe that’s part of the solution, not the problem. I already see people including me going back to real world, local interactions and connections.

It is already a problem. Try interviewing people from LinkedIn and you'll face an onslaught of imposters. https://www.darkreading.com/remote-workforce/north-korean-op...

If anything deepfakes will be good for the economy because if you can’t do business with people who are far away it becomes harder to outsource.

> damage it will cause to the economy when you can no longer trust that you're on a video call with an actual person

What damage are you talking about?

I'm not sure I understand why it matters that there is no real person there if you can't actually tell the difference. You're just demonstrating that you don't actually need a human for whatever it is you're doing.

only proves you're not a corporate model rather than locally running model that's been trained to allow saying that

This was a natural thing to try so I did and even Grok will simply obey instructions to say all those. You don't need one of those ablated open models.

Don’t forget Tiananmen Square to catch the Chinese models.

Where are the em dashes, "octopoc"?

Yes, this was exactly my thought. The caveat is, the phrases that most models refuse to say are the phrases that most people don't want to hear.

That's a bargain Johnny boy! My company gives me $250 in AI tokens to use every day!

This phenomenon (or a closely related one?) is recognized and known as Kotov Sydnrome in the context of chess.

A summary, courtesy of chess dot com:

> The name of this "syndrome" comes from GM Alexander Kotov, author of the classic chess book Think Like a Grandmaster. In the book, Kotov described an incorrect yet very common calculation process that often leads players to select a suboptimal or bad move.

> According to Kotov, in positions where the lines are complex and there are numerous candidate moves and variations to calculate, it's easy to make a hasty move. A player in that situation might spend too much time going over two moves and all of their ramifications without finding a favorable ending position. In that process, the player is likely to go back and forth between the two different lines, always coming to the same unsatisfying conclusion—this wastes precious mental energy and time.

> After spending too much time evaluating the first two options, the player gives up the calculation due to time pressure or fatigue and plays a third move without calculating it. According to the author, that sort of move can cause tremendous blunders and cost the game.

Given enough time to reconsider options, people will be endlessly flip-flopping between them grabbing onto various features over and over in a loop.

People will default to believing something is AI if there's no downside to that opinion. It's a defence mechanism. It stops them being 'caught out' or tricked into believing something that's not true.

As soon as there's a potential loss (e.g. missing out on getting rich, not helping a loved one) people will switch off that cynical critical thinking and just fall for AI-driven scams.

This is the downside of being a human being.

Dissonance between what you instinctively believe and what you think the other person wants you to say.

Easy to replicate by asking someone something obvious, like the weather, and when they reply ask “are you sure?” - they won’t be so sure any more (believing it’s a trick question)

If I ask my mother if I’m real, she’ll have a pause because she has never had to entertain such a question, or the possibility her son over the phone is an impostor. Good way to push someone towards paranoia and psychosis.

I have a systematic way of approaching this kind of situation, where you have to rapidly estimate a thing, commit to the estimate and are judged by the quality of your estimates in the long run; my approach is to first make a guess based off my gut, and then to pause and make a bet with myself, did I guess high or low? If my gut then says that my first gut instinct was too high or low, I adjust from there. I can't guess great the first time, but this two-stage guessing works a lot better for me.

I'm sure I'm not the first to use this technique, but I don't know what it's called.

Paradox of choice? It's more related to the number of choices and the impact on people's anxiety, but it's close.

Analysis Paralysis?

There's also another phenomenon which is that whatever the latest idea is, it must be the best. Many people do this mistake and even convince themselves of being right now because "they used to think like that" before.

So at each stage in the loop they are always super convinced of the position.

I bet that a confident scammer is prepared to deal with things like that. They want to put you in a state where you are under time and emotional pressure and your "relative" will have a well practiced response why they can't answer your weird questions.

Imagine your crying grandson who caused a traffic accident in Mexico and the police planted drugs in his car and now he needs money to pay them off. He is in pain and probably has a concussion (explanation why he can't remember what you are asking), the police is hassling him to get off the phone (time pressure, explanation why the quality of the call is terrible). Will you get hung up on some code word he asked you to memorise years ago and you can't even know where it is anymore? And if you bring it up he just starts crying and tells you that you are his last chance to turn his life around. And you remember when he was a wee little kid and he fell and scraped his knee and you comforted him. Just the thought of pressing him on the code makes you feel like a terrible person. Or not. And then the scammer just finds someone more gullible. Theirs is a number game after all.

Or just find a shared memory/moment not available on the internet when in doubt. I don't think people will be that eager to remember another passphrase.

A password, you mean?

We have two for our alarm system, a shibboleth and a duress word. You write yours a card and seal the envelope and it's couriered to the operators.

Being sufficiently paranoid, the second you use such a phrase you've shared it. Someone will be listening in, and it's only a question of time until criminals get their mitts on that data in the next breach.

Here's also a nice project, the "Reverse Turing Test":

https://ars.electronica.art/panic/de/view/reverse-turing-tes...

(I.e. trying to hide the fact that you're human, among a group of AIs)

One was more parameters, sure.

More training on fingers specifically.

Image VAEs (variation auto encoders) are functions that compress the latent (working) image down. The earlier VAEs would mess up fine details. At a most basic level, just picture compression issues.

Training against bad previous work with six fingers.

Models working in 1024 instead of 512.

Recently one of my friends got email hijacked and whatever entity it was seemingly used her past sent emails as a training corpus to construct some very convincing pleas for donations involving a dog rescue she's been operating for several years.

It also included personal details only her closest friends and family would know. I assume this is being done at scale now. These are NOT Nigerian prince scams of yesteryear; this is something entirely different.

I think this is naive, is it just kicks the can. How do you trust that the signer is human?

The sign/countersign still works even if it's unilateral. You say "the migrating birds fly low over the sea", they say "I told you already, we're not doing this stupid thing", and now they are authenticated.

The only companies that'd follow the watermark are the good guys though, yeah?

The people you'd want to be wary of would be the ones that'd look legit.

e.g. "yes i guess i will send my son $400,000 in cash tonight because he's been kidnapped, and i know it's real because there's no AI watermark that all the nice US/EU companies use."

Yes. And I’m here for it.

Necessity is the mother of invention.

It’s absolutely asinine that we’re still relying on paper birth certificates and social security numbers, and stupid tax systems. I’m interested in breaking everything we have to see what comes next.

I can already see the Nextdoor post: "Watch out for this man who is knocking doors around 10th street! He knocked on mine claiming to be my nephew and even looked the part. Already called the police but they arrived late."

Did Treyvon make it home safely after arguing with George Zimmerman, and then intentionally leave his home to go start a fight on the bad advice of his girlfriend to [not get punked]?

> liberal arts annotator

Tell us more about this axe you appear to need to grind.

How the hell is "he's real" whitewashing.

I heard that in France, they'd use postal office workers to verify people's IDs. It's a brilliant alternative to whatever we're doing in Germany.

Today, we proudly announce, the Meta Rayban 365

The bus that couldn’t slow down.

Or in general, a way to digitally sign a tamper-free video recoding made with a camera from a reputable manufacturer. Maybe a regular iPhone already has enough integrity checks and security contexts to achieve this.

How exactly would this make money

We used to have something similar in NZ. Got removed eventually because of flashing.