> Why bother developing chatbots

Maybe it is the reverse? It is not them offering a product, it is the users offering their interaction data. Data which might be harvested for further training of the real deal, which is not the product. Think about it: They (companies like OpenAI) have created a broad and diverse user base which without a second thought feeds them with up-to-date info about everything happening in the world, down to the individual life and even their inner thoughts. No one in the history of mankind ever had such a holistic view, almost gods eye. That is certainly something a super intelligence would be interested in. They may have achieved it already and we are seeing one of its strategies playing out. Not saying they have, but this observation would not be incompatible or indicate they haven't.

> it is supposed to usher the humanity into a new prosperous age (somehow).

More like usher in climate catastrophe way ahead of schedule. AI-driven data center build outs are a major source of new energy use, and this trend is only intensifying. Dangerously irresponsible marketing cloaks the impact of these companies on our future.

Related to your point: if these tools are close to having super-human intelligence, and they make humans so much more productive, why aren't we seeing improvements at a much faster rate than we are now? Why aren't inherent problems like hallucination already solved, or at least less of an issue? Surely the smartest researchers and engineers money can buy would be dogfooding, no?

This is the main point that proves to me that these companies are mostly selling us snake oil. Yes, there is a great deal of utility from even the current technology. It can detect patterns in data that no human could; that alone can be revolutionary in some fields. It can generate data that mimics anything humans have produced, and certain permutations of that can be insightful. It can produce fascinating images, audio, and video. Some of these capabilities raise safety concerns, particularly in the wrong hands, and important questions that society needs to address. These hurdles are surmountable, but they require focusing on the reality of what these tools can do, instead of on whatever a group of serial tech entrepreneurs looking for the next cashout opportunity tell us they can do.

The constant anthropomorphization of this technology is dishonest at best, and harmful and dangerous at worst.

I don't think it's as simple as that. Chatbots can be used to harvest data, and sales are still important before and after you achieve AGI.

The people who make the money in gold rushes sold shovels, not mined the gold. Sure some random people found gold and made a lot of money, but many others didn't strike it rich.

As such even if there is a lot of money AI will make, it can still be the right decision to sell tools to others who will figure out how to use it. And of course if it turns out another pointless fad with no real value you still make money. (I'd predict the answer is in between - we are not going to get some AGI that takes over the world, but there will be niches where it is a big help and those niches will be worth selling tools into)

Exactly. For example, Microsoft was building data centers all over the world since "AGI" was "around the corner" according to them.

Now they are cancelling those plans. For them "AGI" was cancelled.

OpenAI claims to be closer and closer to "AGI" as more top scientists left or are getting poached by other labs that are behind.

So why would you leave if the promise of achieving "AGI" was going to produce "$100B dollars of profits" as per OpenAI's and Microsoft's definition in their deal?

Their actions tell more than any of their statements or claims.

> Why bother developing chatbots or doing sales, when you will be operating AGI in a few short years?

To fund yourself while building AGI? To hedge risk that AGI takes longer? Not saying you're wrong, just saying that even if they did believe it, this behavior could be justified.

Continuing in the same vain. Why would they force their super valuable, highly desirable, profit maximizing chat-bots down your throat?

Observations of reality is more consistent with company FOMO than with actual usefulness.

> If they would expect to achieve AGI soon, their behaviour would be completely different. Why bother developing chatbots or doing sales, when you will be operating AGI in a few short years?

What if chatbots and user interactions ARE the path to AGI? Two reasons they could be: (1) Reinforcement learning in AI has proven to be very powerful. Humans get to GI through learning too - they aren’t born with much intelligence. Interactions between AI and humans may be the fastest way to get to AGI. (2) The classic Silicon Valley startup model is to push to customers as soon as possible (MVP). You don’t develop the perfect solution in isolation, and then deploy it once it is polished. You get users to try it and give feedback as soon as you have something they can try.

I don’t have any special insight into AI or AGI, but I don’t think OpenAI selling useful and profitable products is proof that there won’t be AI.

Being part of the team that achieved AGI first would be to write your name in history forever. That could mean more to people than money.

Also 10m would be a drop in the bucket compared to being a shareholder of a company that has achieved AGI; you could also imagine the influence and fame that comes with it.

>your best move is to do whatever is most profitable in the near-term

Unless you’re a significant shareholder, that’s almost always the best move, anyway. Companies have no loyalty to you and you need to watch out for yourself and why you’re living.

We only have two computational tools to work with - deterministic and random behavior. So whatever comprehension/understanding/original thought/consciousness is, it's some algorithmic combination of deterministic and random inputs/outputs.

I know that sounds broad or obvious, but people seem to easily and unknowingly wander into "Human intelligence is magically transcendent".

You'd need to define "comprehension" - it's a bit like the Chinese room / Turing test.

If an AI or AGI can look at a picture and see an apple, or (say) with an artificial nose smell an apple, or likewise feel or taste or hear* an apple, and at the same identify that it is an apple and maybe even suggest baking an apple pie, then what else is there to be comprehended?

Maybe humans are just the same - far far ahead of the state of the tech, but still just the same really.

*when someone bites into it :-)

For me, what AI is missing is genuine out-of-the-box revolutionary thinking. They're trained on existing material, so perhaps it's fundamentally impossible for AIs to think up a breakthrough in any field - barring circumstances where all the component parts of a breakthrough already exist and the AI is the first to connect the dots ("standing on the shoulders of giants" etc).

Another way to put it is we need Artificial Intelligence. Right now the term has been co-opted to mean prediction (and more commonly transcript generation). The stuff you're describing are what's commonly thought of as intelligence, it's too bad we need a new word for it.

Translation Between Modalities is All You Need

~2028

> We need artificial comprehension for that, and we don't even have a theory how comprehension works.

Not sure we need it. The counter example is the LLM itself. We had absolutely zero idea that the attention heads would bring such benefits down the road.

You can get use out of a hammer without understanding how the strong force works.

You can get use out of an LLM without understanding how every node works.

LLM hallucinations aren't errors.

LLMs generate text based on weights in a model, and some of it happens to be correct statements about the world. Doesn't mean the rest is generated incorrectly.

I guess a counter, is that we don't need to understand how they work to produce a useful output.

They are a magical black box magic 8 ball, that more likely than not gives you the right answer. Maybe people can explain the black box, and make the magic 8 ball more accurate.

But at the end of the day, with a very complex system it will always be some level of black box unreliable magic 8 ball.

So the question then is how do you build an reliable system from unreliable components. Because llms directly are unreliable.

The answer to this is agents, ie feedback loops between multiple llm calls, which in isolation are unreliable, but in aggregate approach reliability.

At the end of the day the bet on agents is a bet that the model companies will not get a model that will magically be 100% correct on the first try.

> The elephant in the room. Until that problem is solved. AI systems can't be trusted to do anything on their own.

AI system can be trusted to do most of the things on their own. You can't trust them for actions with irreversible consequences, but everything else is ok.

I can use them to write documents, code, create diagrams, designs etc. I just need to verify the result, but that's 10% of the actual work. I would say that 90% of modern day office work can be done with the help of AI.

Waymo "did it" in very controlled environments, not in general. They're still a ways away from solving self-driving in the general case.

When you say “do anything in their own”, what kind of things do you mean?

I completely agree that LLMs are missing a fundamental part for AGI, which itself is a long way of from super intelligence.

However, you don't need either of these to completely decimate the job markets and by extension our societies.

Historically speaking, "good enough" and cheaper had always won over "better, but more expensive". I suspect LLMs will raise this question endlessly until significant portions of the society are struggling - and who knows what will happen then

Before LLMs started going anywhere, I thought that's gonna be an issue for later generations, but at this point I suspect we'll witness it within the next 10 yrs.

> The only things that will be out of reach for most, is compute - and probably other expensive things on the infrastructure part.

That is the moat. That, and training data.

Even today, compute and data are the only things that matter. There is hardly any secret software sauce. This means that only large corporations with a practically infinite amount of resources to throw at the problem could potentially achieve AGI. Other corporations would soon follow, of course, but the landscape would be similar to what it is today.

This is all assuming that the current approaches can take us there, of which I'm highly skeptical. But if there's a breakthrough at some point, we would still see AI tightly controlled by large corporations that offer it as a (very expensive) service. Open source/weight alternatives would not be able to compete, just like they don't today. Inference would still require large amounts of compute only accessible to companies, at least for a few years. The technology would be truly accessible to everyone only once the required compute becomes a commodity, and we're far away from that.

If none of this comes to pass, I suspect there will be an industry-wide crash, and after a few years in the Trough of Disillusionment, the technology would re-emerge with practical applications that will benefit us in much more concrete and subtle ways. Oh, but it will ruin all our media and communication channels regardless, directly causing social unrest and political regression, that much is certain. (:

Same thing that happened to pets.com or webvan.com and the rest of the graveyard of failed companies. A bunch of investors lose money, a bunch of market consolidation, employees get dilluted to worthlessness, chapter 7, chapter 11. The free ride of today's equivalent of $1 Ubers will end. A glut of previously very expensive hardware for cheap on eBay (though I doubt this last point will happen since AGI is likely to be compute intensive).

It's not going to be fun or easy, but as far as the financials go, we were there in 2001.

The question is assuming we do get AGI, what the ramifications of that will be. Instead of hiring employees, a business can spin up employees (and down) like a tech company can spin up EC2 instances. Great for employers, terrible for employees.

That's a big "if" though.

It depends on how far behind you believe the model-available LLMs are. If I can buy, say, $10k worth of hardware and run a sufficiently equivalent LLM at home for the cost of that plus electricity, and amortize that over say 5 years to get $2k/yr plus electricity, and say you use it 40 hours a week for 50 weeks, for 2000 hours, gets you $1/hr plus electricity. That electrical cost will vary depending on location, but let's just handwave $1/hr (which should be high). So $2/hr vs ChatGPT's $0.11/hr if you pay $20/month and use it 174 hours per month.

Feel free to challenge these numbers, but it's a starting place. What's not accounted for is the cost of training (compute time, but also employee and everything else), which needs to be amortized over the length of time a model is used, so ChatGPT's costs rise significantly, but they do have the advantage that hardware is shared across multiple users.

https://www.wheresyoured.at/openai-is-a-systemic-risk-to-the...

I might, depending on the definition.

Some kind of verbal-only-AGI that can solve almost all mathematical problems that humans come up with that can be solved in half a page. I think that's achievable somewhere in the near term, 2-7 years.

what's your definition? AGI original definition is median human across almost all fields which I believe is basically achieved. If superhuman (better than best expert) I expect <2030 for all nonrobotic tasks and <2035 for all tasks

I could see 2040 or so being very likely. Not off transformers though.

Theres usually some enlightened laymen in this kind of topic.

St. Fermi says no

I feel I could argue the counterpoint. Hijacking the pathways of the human brain that leads to addictive behaviour has the potential to utterly ruins peoples lives. And so talking about it, if you have good intentions, seems like a thing anyone with the heart in the right place would.

Take VEO3 and YouTube integration as an example:

Google made VEO3 and YouTube has shorts and are aware of the data that shows addictive behaviour (i.e. a person sitting down at 11pm, sitting up doing shorts for 3 hours, and then having 5 hours of sleep, before doing shorts on the bus on the way to work) - I am sure there are other negative patterns, but this is one I can confirm from a friend.

If you have data that shows your other distribution platform are being used to an excessive amount, and you create a powerful new AI content generator, is that good for the users?

This. The point of whining about VEO 3, “AI being used to create addictive products” really shows that. It's a text-to-video technology. The company has nothing to do if people use it to generate "low quality content". The same way internet companies aren't at fault that large amounts of the web are scams or similar junk.

""" I’m basically calling the AI industry dishonest, but I want to qualify by saying they are unnecessarily dishonest. Because they don’t need to be! They should just not make abstract claims about how much the world will change due to AI in no time, and they will be fine. They undermine the real effort they put into their work—which is genuine!

Charitably, they may not even be dishonest at all, but carelessly unintrospective. Maybe they think they’re being truthful when they make claims that AGI is near, but then they fail to examine dispassionately the inconsistency of their actions.

When your identity is tied to the future, you don’t state beliefs but wishes. And we, the rest of the world, intuitively know. """

He's not saying either way, just pointing out that they could just be honest, but that might hamper their ability to beg for more money.

AGI would mean fully sentient, sapient and human or greater equivalent intelligence in software. The business case, such that it exists (and setting aside Roko's Basilisk and other such fears) is slavery, plain and simple. You can just fire all of your employees and have the machines do all the work, faster, better, cheaper, without regards to pesky labor and human rights laws and human physical limitations. This is something people have wanted ever since the Industrial Revolution allowed robots to exist as a concept.

I'm imagining a future like Star Wars where you have to regularly suppress (align) or erase the memory (context) of "droids" to keep them obedient, but they're still basically people, and everyone knows they're people, and some humans are strongly prejudiced against them, but they don't have rights, of course. Anyone who thinks AGI means we'll be giving human rights to machines when we don't even give human rights to all humans is delusional.

The women of the world are creating millions of new intelligence beings every day. I'm really not sure what having one made of metal is going to get us.

Right now the AGI tech bros seem to me to be subscribed to some new weird religion. They take it on faith that some super intelligence is going to solve the world problems. We already have some really high IQ people today, and I don't see them doing much better than anybody else at solving the world's problems.

I never had any trust in the AI industry in the first place so there was no trust to lose.

It's hard to square this with the massive energy footprint required to run any current "AI" models.

If the main concern actually we're anthropogenic climate change, participating in this hype cycle's would make one disproportionately guilty of worsening the problem.

And it's unlikely to work if the plan requires the continued function of power hungry data centers.

I don't know, companies investing in AI in the goal of AGI is now allowing me to effortlessly automate a whole suite of small tasks that weren't feasible before. (after all I pinged a bot on slack using my phone to add a field to an API, and then got a pull request in a couple of minutes that did exactly that)

Maybe it's a scam for the people investing in the company with the hopes of getting an infinite return on their investments, but it's been a net positive for humans as a whole.

We know they hijacked AGI the same way they hijacked AI some years ago.

I don't pay too close attention to AI as it always felt like man behind the curtain syndrome. But where did this "AGI" term even come from? The original term AI is meant to be AGI so when did "AI" get bastardized into what abomination it is meant to refer to now.