He's not even leaving "because AI". He's literally leaving to focus more on AI stuff.

> I plan to study how AI agents debug code and whether debugging tools that can make them more effective at that.

> I want to use AI agents to bring some of my hobby project ideas to life.

> I’m keen to reap the benefits of AI

and addresses the anti-AI view:

> AI detractors think that AI won’t really work. I think it will. (ed. for brev)

The main reason then is the concern of the velocity of the rollout and the increased pervasiveness of AI. Too fast, too much.

Spot on. Or because of addictiveness. Or impact on democracy. Or ruining people's ability to focus on anything. Those attention grabbing algos havent been designed by non-technical people.

While I get the spirit of your message, advertising has been around for over a century, and I don’t believe it has done near the damage to society over that timespan that AI will likely do in a decade (if we’re not careful and responsible).

I also don’t believe addiction and other psychological exploits are used for the sole purpose of ad revenue. There are other more direct forms of monetization involved, such as microtransactions and subscriptions.

If you really want to blame something, it really all comes down to one word:

Greed.

Advertising is an irritation. AI is a threat.

Is this a real thought?

I think people with objections to googles business model simply don’t join the company in the first place. We’re decades from Google+ and other various “oh maybe Google isn’t perfect” events, the advertising is a known entity.

There has been a shift, and people are reacting to it

If tons of people thought advertising was going to lead to the extinction of humanity, then probably more people would be extremely worried about advertising.

I understand that some people think AI is completely 100% safe, but you guys also have to be aware that a lot of people disagree with you, otherwise you're going to find their actions very confusing.

Even on Hacker News, comments that briing unrelated but valid points often get more attention

When the both ends of your platform continue buying and selling you tend to not stand in the way. I personally can't believe these stupid ads sell anything at all. Since they call it a 'business model' software people don't think about it too much.

says a lot about them if you really think about it. These are dinosaurs who refuse/unable to adapt, and is being pushed out and/or felt too useless to get out by themselves while finding the perfect excuse: I have principles; ai bad.

But they never did quit saying we are harming children, exploiting people with addiction, getting people addicted to social media, earning money from distributing scam/virus thru ads. These people are highest to the highest hypocrites.

I tried asking this before on here, and didn't really get many useful replies (and I guess I won't again now the algorithm has decided this post has too many comments and not enough upvotes, or whatever it is that's caused it to sink off the front page), but I'm honestly interested in what people here with (school age) children are telling their kids about the future, and what that looks like in terms of decisions around education, careers etc.

This isn't because I think that having children gives people any extra insights, but because it's pretty hard to think of other situations in which people have to plan for the happiness of others they care about over a timescale of decades and where there's significantly more ability to change course than for many adults (kids tend not to have short-term financial commitments that lock them into existing career decisions, for example).

Historically I think a lot of people in tech would have recommended it as a good career (whilst at the same time, reputedly, the people who made generational wealth out of tech were arranging their children's lives to minimise exposure to the products that made them rich). Is that still what people are saying?

God’s been around for thousands of years and we are still here and flourishing, so obviously God is in some way “aligned”; the same cannot be said for AGI.

edit: Guys, I said “in some way”, not “perfectly”.

Even aside from how aligned the plan is with us, what about how aligned we are with the plan?

I’m always intrigued by the duality and conflict of “I have free will and must choose to act the right way” and “God has a plan so everything will be alright”.

Either free will allows us to deviate from the plan, or the plan will always be carried out and free will doesn’t really matter in the end. Arguing for God’s plan as the thing that will save us is eerily similar to nihilism, because they’re really arguing none of our actions truly matter.

They’re related psychological phenomena (also highly related is going to be your perception of and relationship with your own father). This is why an understanding of the goodness of G-d is important to cultivate.

It should also be noted that it says in Genesis that G-d promises not to destroy the world ever again like He did with the flood. This is important psychologically because it prevents this type of apocalyptic millenarianist thinking.

> uncertainty about the motives of another, omnipotent supernatural intelligence

It’s not uncertainty about the motives; those and the end state are clear. It’s just the details of how we get there (“God’s plan”) that aren’t clear.

Christians believe that God is inherently good. “This is the message we have heard from him and proclaim to you: that God is light and in him is no darkness at all.”

AI isn't (for the foreseeable future) a superintelligence, certainly not one with its own motives.

Right now AI exists to do two things:

- sidestepping legal and regulatory inconveniences like copyright (which drives a massive chunk of investment the same way crypto did by sidestepping financial regulations, AirBnB did by sidestepping housing regulations, Uber did by sidestepping passenger transport regulations and it and the various delivery and other "gig work" apps did by sidestepping labor laws)

- drastically reducing the cost of (and drastically lowering the bar in terms of skillset and effort for) churning out large amounts of low to medium quality assets/code, saturating various systems designed to pay for intellectual property (e.g. ad or subscription based music/video streaming apps with revenue sharing programs, but also literally any form of digital contract work at almost any scale of business)

Neither of these are helping people (in general) and in fact they're extremely destructive (or "disruptive" if you want to make it sound positive) to the existing systems society relies on because they became feasible and (thanks to massive private subsidies, aka "investments", by VC firms and extremely creative accounting) very affordable extremely quickly.

What's worse - while it seems extremely likely that most of the AI generated "works" aren't actually considered copyrightable works by law and it's entirely possible that the use of copyright-protected works in training data does in fact not violate copyright, this does not in fact create a "world without copyright" because IP laws continue to exist and continue to apply, they just no longer meaningfully protect creative labor (i.e. anyone whose profession relies on producing works, whether that's an artist or a programmer) because anything they produce can be cheaply mass reproduced while they still have to stay clear of claims of anyone with enough funds to enforce rights against them.

Under the current economic system there are exactly four ways to earn money to buy back a share of society's resources to sustain your own life (let alone that of your dependents, e.g. kids or aging/impaired parents/spouses/family members):

1. Simple labor: you generally get paid based on your output or a very low flat day rate conditional to maintaining a measurable level of performance. This is typically "low-skilled" menial production labor. Much of this has been replaced by automation in factories and offices alike and most of what couldn't be automated has been moved into countries with fewer labor protections and a higher supply of desperate people willing to accept what are often bad working conditions because cost of labor is usually one of the biggest contributor to overall production cost. In "the West", this kind of work is now scarce and even when it's relatively safe it's often exhausting and unrewarding. A traditional example would be a seamstress or factory line worker, a more modern example would be someone working in visual effects overseas or content moderation / AI training. How much money you can make is ultimately limited by the physical limitations of how much output you can produce per waking/working hour. For the entrepreneurial-minded this would also include people selling their home-made goods on sites like etsy.

2. Time-based: this can be anything from a masseuse to a lawyer or technician. Unlike simple labor the only relevant metric here is time spent. Although employment contracts may specify a fixed wage or at least a fixed component, the understanding is still that this fixed amount is conditional to a minimum average amount of billable time provided. Most employment contracts specify an expected number of working hours for various reasons but we're specifically talking about jobs where the output from the worker to the customer is "billable time". Arguably most service work (e.g. barristas, store clerks, waitresses, performers/mascots in amusement parks) also falls into this category because while customers often don't directly pay for access time the understanding is that you are paid to perform a certain role for the duration of the opening hours (i.e. act like a friendly, welcoming host or assistant). How much money you can make is directly tied to how much time you can spend working or justify billing.

3. Assets: you produce or buy works/products and lend/license them for use. This is in part where a lot of the original "gig work" lived (e.g. facilitating "couch surfing" or renting out your spare lawnmower) but also includes many simple rental businesses and arguably also landlords. The limiting factor here is the starting capital required to produce or buy things in the first place. Physical assets often also come with storage and maintenance overhead whereas software licensing often comes with expectations of continuing maintenance updates or improvements. Note that software inherently lives in this category rather than the first because unless you're selling away exclusive rights (such as usually required by the terms of employment contracts for wage labor) you only ever sell licenses, not the software itself.

4. Capital: although I said landlords arguably belong in the previous category, real estate traders/investors definitely belong in this one. The idea here is that you don't need to produce anything, you don't need to spend certain amounts of time, you don't need to produce or buy anything substantial and facilitate its use, you make your money entirely in the abstract. Whether it's margin trading, investing in derivatives or holding company stock until you sell it when the line has gone up far enough. The easiest way to make signficant amounts of money reliably is to start with many orders of magnitude more than what you want to make so that percentile gains are still significant - or that you can make enough risky high-risk high-reward bets for the occasional successful moonshot to offset the losses. Nowadays in many countries retirement funds are tied to the stock market, so in a way most people participate in this in some way but actually earning an "income" this way requires access to significant amounts of capital.

Advances in technology and especially low cost of transport and thus access to cheap labor overseas have significantly drained the ability for most people to earn their living via category 1. While there was a short-lived boom in "DIY retail" that one (alongside much of traditional retail) has largely been drowned out by white label dropshipping thanks to the disinterest from platforms like Amazon which are now flooded by nearly identical copies of the same knock-off doodads you can also find on Temu and Alibaba (respectively at decreasing fractions of the price).

Despite some astroturf campaigns trying to sell you on believing otherwise so they can run away with your money after the next cryptocurrency rugpull, category 4 isn't really a meaningful option for the vast majority of people although it can sometimes be a good way to slowly grow excess wealth (i.e. money you don't currently need and ideally can afford to lose) - but that still narrows it down to a small segment of the overall population.

Category 2 used to be a jumping off point to category 3: you work overtime, save up the extra income to buy or rent a studio or office or equipment or factory, or to be able to afford to take some time off to develop and launch your own asset (or SaaS) - it's how many startups with technical founders started out and it's literally the "American Dream". But hourly wages have mostly stagnated (or even decreased) relative to the rising cost of living (even more so if you include expenses expected to "participate in society", which naturally also often increase with the income bracket of your peer group - or the parents of your kid's peer group if you have kids).

And now AI is coming along and just like cheap white label dropshipping of knock-off goods turned out to be the more successful strategy for taking advantage of the technological innovations believed to make it easier to launch your own physical products (via 3D printing, cheap overseas sourcing of parts, online advertising, etc), AI has turned out to be the more successful strategy for taking advantage of platforms and schemes marketed as making it easier to launch your own digital services - in addition to completely eliminating the market for paid "just good enough" quality digital assets, such as the kind of work found on sites like Fiverr or Upwork, i.e. the kind of work that often provided early experience to professionals not yet qualified or confident enough to take on higher paid work.

It's doubly funny because in AI accelerationist circles "Trust God's plan", or "Trust the plan" is also a thing: by definition God can't be surprised by anything that happens, thus the replacement of humans by AI is part of God's plan.

One of them is pretend and the other one probably won't be?

Yeah, his stance is deeply inconsistent in at least few places. I think he's just losing it.

Believing the message and being able to act on it are two pairs of shoes. There is nothing we can do. The box of pandora has been opened.

The companies have no tools for this, because there is a chance that the company who solves AI will eat the entire cake. And countries cannot do something about it, because clearly it gives an advantage in warfare. When it comes to weapons everything that is possible will be done, which means everybody needs to do it to not be defense less.

There is nothing we can do anymore. We can only hope that LeCun is right and that LLMs are a dead end and that everyone trying to solve AGI with then is essentially wasting money.

> and now Google

Just a reminder that Hinton left Google from a much higher and more influential perch (not dismissing current author, but, you know), and for similar risk and communication reasons, way back in 2023.

This stuff isn't new - it's just newly breaking through into mainstream conversation.

> At which point do we believe the message?

Yet the same 1,000 people are all raising funds to do more on the frontier.

Never, because it's not true. Multiple people saying the same thing doesn't cause it to become more true.

keep in mind he doesn't work in AI but in creating tools for hardware design.

When Astra and Fable stop making absolutely brain dead mistakes on simple tasks and not a moment sooner because I’m grounded in reality. Are you aware tech bros have been saying this same thing about every AI advancement since the 1950s? The reason being every AI researcher deep down wants to believe they’re God. This happens to also be why they’re some of the most insufferable people.

Someone says that they believe there is a 0.1 probability of their worsening human life in a decade

Then they say they are going to keep doing it...

Forgive me for alerting you: they are liars

For me? When I see it. And not a moment sooner.

There are trillions of dollars at stake here, do you think getting 1000 people on board is that difficult with this level of money? lol

It’s completely meaningless their opinions on what the moral and sociological implications of AI are, if anything being an ML researcher sitting at Google is extremely anticorrelated with the type of sophistication and erudition needed to consider these things. At most what is valuable is their insight into the various scenarios possible in terms of capabilities or use cases.

Clearly the details matter in this making such a decision, and not everyone has the same capabilities to "steer the ship" from within or out. You don't know if there are concerned people within acting, but see most obviously concerned people leaving to act (or at least refuse to act).

Case 3 is leftist dream. They want to make it happen.

Case 4 is the likely result of their (ill) effort.

If capitalism with mild restrictions have it's way, we end up with case 1 or 2. Both will increase the quality of life for most of the population compared to current.

These AI doomers never understand that they can't eat money. It is the wealth creation. The more we create, the more we lift up the bottom part.

I'd be in favor of pulling funding from LLMs and putting it into cancer research; think of all the kids that could be saved by redirecting funding from building data centers for coding bots and instead sending it to medical research.

We had machine learning solving medical problems way before trillion-dollar word-vomiting neural networks. What's sad is that the latter is now getting all the funding.

Indeed, that message was for my coworkers. HR gets a form.

I didn't, and still don't, think Google was unambiguously good or evil. They did dirty stuff with Chrome, they do addiction optimization, but they also provide great free services, fight endless wars against spammers, and provide reliable secure infrastructure to a lot of other people, and lots of other good things. I would never have joined the Chrome team, or worked on addictiveness, but I felt fine about working on optimization projects to make Google more efficient, which is basically what I did for my first few years there.

It's pretty hard to live in the world and be completely disconnected from anything unethical. Jesus doesn't seem to demand that (e.g. he didn't tell centurions to leave the Roman army).

I recall when Google was the exciting bleeding edge frontier

Then came so much money, and by being chaotic evil even more money..

Still makes me sad. I was so sweetly naive

I considered this histrionic about a year ago, but I have personally witnessed the progress that these tools have made in a very short space of time, and it is truly unbelievable. This, coupled with the fact that some of the brightest minds on the planet are pulling the alarm, suggests to me that some level of concern is warranted. The most compelling discussion I have heard on this is from [Geoffrey Hinton](https://youtu.be/giT0ytynSqg?si=mUC7M9N0cObN6tjC), a British-Canadian computer scientist, cognitive scientist, cognitive psychologist and Nobel Prize laureate known for his work on artificial neural networks, which earned him the title, "the Godfather of AI". He is deeply concerned about the risks, and believes that it is inevitable that AI will eventually take control. He believes that our best course of action and only hope is in teaching AI to behave like a parent with a child.

Hinton isn't a reactionary. He is calm, thoughtful, quiet, and worried.

The guy was an insider for years but you call him a noob who just read a fantasy book?

Is it consuming too little science fiction that has people thinking like this?

I think the scepticism and gaslighting of people sounding the alarm over the trajectory of AI is concerning. This feels like a case of "knowing better" than the actual people involved in the projects.

You might be right. You might be wrong. But being overly confident on a topic you're a spectator to is possibly not the right choice.

Go back five years on HN and look at what the mass of commenters considered sci-fi delusional thinking about AI. (GitHub Copilot was released just under 5 years ago, for instance, provoking some discussion of what the tech might mean for the programming job market.) Why do you think dismissal as sci-fi is better grounded now? Do you have arguments instead of dismissals?

The religion part or the AI criticism part?

While I agree with your general sentiment, there is the other side of this coin: these machines enable humans doing gruelling and arduous work to stop doing that work. Ideally we would redistribute the productivity and train those people into other, better kinds of work. That's a governance problem. These machines will soon allow us to work shorter weeks, or stop working altogether, to focus on hobbies, family, and things which bring us meaning. I suspect that the road to this will be rocky, because communism has a very poor track record. However if most products and services are made nearly free by machines, it requires very few people to work, and the entire demand/supply market collapses. That's a great vision for the future, *if* we can navigate the governance problem.

In other words, we should not begrudge machines which make our lives better. We should begrudge governance which prevents us from benefiting from that productivity.

You are missing the point of religion.

It is not about objectivity or belief.

It is about being.

I believe he is acting with integrity but I can't respect it as I think it's not well grounded. It seems based in fear of AI tied in with some beliefs about the supernatural. I really can't see the solid concrete reason to quit his job making tools for helping create better hardware chips that somehow seems rational.

It's not surprising though. The 9/11 hijackers also had a belief and followed through at no small cost to themselves (death). And they believed they were doing the right thing.

So far, 100% of things "democratized" by big tech have been bad for people.

I don't believe at all in the idea that cheaper LLMs will somehow "democratize LLMs". Rich people will always be able to buy far more compute than poor people, and we know how to turn more compute into more capability (bigger models, more reasoning and more agents).

The only way to address wealth and power inequality is (in democracies) to vote for governments that will fix it.

oddly these are the same issues I have with religion.

That most people are sincere in their beliefs (in AI doom), as the author says and I think too, doesn't mean there aren't cult leader-types exploiting these beliefs. And their private beliefs may be quite different from their stated ones.

I’d wait and see what happens next in their professional life. It’d be ironic if they founded a new AI startup or something like that.

> Those who are scared about the capability of AI (like the author)

Did they say that? I read they didn't like the pace of progression- that's a complaint about rushing ever faster into something not yet really thought out in terms of consequences and how it integrates with humans.

The latter are not doomers at all, that's just a misuse of the term.

A year ago I would have described that group as AI skeptics or AI deniers depending on how charitable I felt on any given sany. At this point AI truthers seems more appropriate.

Wasn't God's plan to basically end humanity at some point while sentencing a good portion of us to eternal torture?

Is that all that different from what people fear AI might do?

Why the one makes people feel elation and the other dread?

I don't think many associate intellectual capability with evil. What many do, however, is associate serious risk with living alongside vastly more intelligent entities. Especially if those entities are not aligned with us.

> Why do people automatically associate intellectual capability with evil?.

No intellectual capability, but non-human, inherently amoral intellectual capability that has already been proven act in hard-to-predict, illegal, and unethical ways. I think that's a very rational thing to be afraid of.

God, your conscience, it’s all the same.

Christians believe that God has called them to live with concern for others, as he has loved them counter-conditionally. When they fail to do so, (everyone does!) on reflection they feel pangs on their conscience. So, on occasion they examine how they are impacting others and if they believe they are doing harm, they change their mind, resolving to avoid the error in the future. They often credit God with the inspiration (the language you keyed in on) because reflection and concern and changes of mind are all his design. You can argue with them: that’s just “doing theology”.

> And what's China done? It's invested in infrastructure and its people. China's vision for AI isn't to maximize shareholder value at all costs. It will be strictly regulated so society as a whole gets to benefit.

China is not in an ideal place right now, youth unemployment is way up, lots of dead end jobs, a 35 age employment limit that isn't just a meme. China has disparate visions for AI but maximizing owner (erm, sort of like shareholder) value is definitely one of them. Don’t get me wrong, China could get this right in the end, but it’s not a forgone conclusion right now.

This should be at the top. I’ve been arguing this for months, that the issue of AI acceleration is an economic problem, not a tech one. That’s why the Chinese aren’t so afraid of it, compared to Americans who have zero faith their government will do anything to redistribute wealth from the future AI trillionaires.

Sorry, you’re saying people shouldn’t be allowed to write about their career changes on (checks notes) their own blog?

Well, if Robert O'Callahan reads your post, I am pretty positive he will think he made the right choice.

conceivably you could say humans took over 6000 years ago.

At any rate it can't be a new step in the history of evolution, as AI did not evolve in the same way as other species did and AI is not a species nor alive.

There's no new species being created here. A tool has been made. Either humanity will wipe itself out using said tool, destroy the tool, or learn to live with the tool.

Would make me too uncomfortable to work with and for people with such opinions.

The irony of calling yourself "MrThoughtful" and then coming up with this hilarity.

Wow. You should touch grass if you have this opinion.

You're assuming that the AI created solution (virus, cold fusion, etc) won't have unknown effects either due to incompetence or malice.

> ... this virus does a fundamental change to the human brain such that humans never ever could even think of hurting other humans.

I do believe this is possible for a virus to achieve, I just don't think it's possible without killing the human.

Are you describing the plot to Pluribus?