They are currently experimenting with replacing engineers with LLMs, so it would at least be interesting to try the flipped experiment.

It seems like, at least for tasks like distilling the work done at lower levels and passing it up the management tree, LLMs should be good at that. Maybe they could take out some levels at least.

And they can be programmed to be 100% selfless and on-mission: no turf battles or other internal politics. Everybody’s job involves office politics of course, but my gut says management has a lot more of it than engineering, so zeroing that out could be more significant.

Hacker News is awash wirh people who believe many jobs are "bullshit jobs", based on nothing beyond lacking intellectual humility, and ignorance of what the jobs emtsil.

Careful what you wish for…

I wonder if it was (ironically) coined by nonengineers who don't understand how engineers (or other high-stakes professionals at Pareto boundaries) solve problems: by decomposing problems and identifying optional solutions and weighing the solutions based on various risks, trade-offs and tolerances.

I have seen this personally, where merely identifying a possible solution (harmless, reversible) is misinterpreted as selecting and implementing that solution (may be irreversible).

I mean, if one has a solid physics + engineering background, he is potentially equipped to solve many problems from other fields.

I've seen people saying that with other technologies. Years later, they are still in the same job but a lot richer.

If Karpathy's "auto-research" is at all representative of typical "AI research", then presumably it will!

> The AI researchers I know often say they expect the AI to replace their own jobs.

Are they expecting the equity they get in some AI company will save them personally from economic ruin, so they're fine with that outcome for everyone else?

Or are they expecting AI technology will magically make our society communist?

> Anyone remember when YC declared they would replace Hollywood?

I specifically remember a prolific HN commenter working on an AI video startup claiming that by the end of 2025, children would be able to create Hollywood-quality, feature-length movies using available AI video tools.

I believe this person's startup has (wisely) pivoted to building AI-powered tools for the movie industry.

AI is just another tool in the arsenal of tools for both developers (Linus [1]) and movie makers (Ben Affleck [2]). These are expert practitioners in their fields intelligently opining on AI, and they are promoting it not as a magic machine, but as another tool in the toolbox.

[1] https://www.youtube.com/watch?v=3NSSGt9bZag

[2] https://www.youtube.com/watch?v=O-2OsvVJC0s

they might manage to get a semi passable AI movie... but will people pay to watch it? I certainly would not.

I felt this watching the recent season of Clarksons Farm which had some nice product placement for what looked like automation startups. It was hard to watch as a software person as these farmers see the best part of their jobs automated away, but there’s also the rub: nobody wants to spend on automating that part. Eventually economics will require them to but perhaps the way to get started today is to focus on the stuff they actually don’t like doing.

Reminds me of the engineers/designers who regularly announce they've created a new revolutionary form of bicycle. Turns out the bicycle is a (nearly) perfect design and making it more complicated and expensive and less efficient is not an improvement.

That probably should be "meaningfully weigh in" or something. I don't even remember most of the things I "weighed in on" on the web, but everybody'll tell ya, they'll say, he had the least amount of clue possible in any of these things, they actually didn't think it's possible but it's true. But then they realised that even a coin toss gets it right more often than that so they said, wait a minute, this is real, this is happening, this is actually what's going on and they were all amazed because it had never been seen before. So now I'm wired to this flock of ducks that goes into the opposite direction of where I want to go to find out the scientifically best spot to be in at any given moment.

These kinds of late design decisions can cost a huge amount of money. And it may cost lives because it's now shoe horned in, less reliable.

Before OpenAI / situational-awareness he made a lot of money on societal-shift type investments and shorts during the lead up to Covid economic impacts.

His "main thing" is success in calling economic impacts of undervalued large shifts, and the premise of the fund is basically that the same thing is occurring around AI, where he also has specific subject matter expertise.

> generational foresight not available to 99.999999999999999999% of 25 year olds. I want to get ahead of the default cynicism and "oh it's just dumb luck woe is me" aspect of HN.

And I want to get ahead of this absurd romanticism. I have plenty of counterpoints in all types of technical domains (3D printing, blockchain, you name it). Oracles don't exist, just stories of right place right time survivors.

> That's what he based the fund on.

His thesis was so true that his fund folded. Fancy that.

Wait, is this the group who predicted that we'd have expontential AI progress and AGI in 2027 because we'd have AIs training AIs? Predicated on the preposterous assumption that R&DEffort == RateOfProgress. The more sober and realistic assessment is that R&DEffort >= RateOfProgress, with only short bursts of achieving the upper bound. Leave it to a brilliant 25 year old to overfit to one of those short bursts of progress and then assume it will continue forever just as it always has.

What actually happened is that AI-assisted training of LLMs (mind you -- stuck on that same old transformer architecture, with nothing resembling a medium-term memory, and fine-tuning still doesn't really work as a form of "learning") is right now giving us ~linear-ish progress... beecause we already hit the slowdown in the S curve of what human researchers can achieve.

We are also energy-constrained in ways that the exponential forecasts have no answer for other. Compute is not getting more efficient fast enough to achieve anything close to exponential growth driven by growth in computing power. There are social, moral, and political limits on the amount of energy we can dedicate to AI training in any given time period. And it's not just energy, we're short on RAM wafers and water for cooling and eventually we're going to hit other limits on various minerals and elements of the supply chain etc. etc. So even if it were true that R&DEffort == RateOfProgress without resource constraints, we're likely not going to see that exponential progress because we also need corresponding exponential scifi-scale progress in computing efficiency, cooling, and, energy delivery.

Basically this kid made a bunch of very clever and grand predictions which were always kind of ridiculous and no, they have not come to pass in 2026 and we are not at all on track to achieve AGI in the next 12 months, despite what Altman keeps trying to make the public think.

It’s pretty standard insight among the machine learning crowd. This kid founded the effective altruist club at his school. He’s clearly very smart and ambitious, and has leadership qualities, but you don’t have to exaggerate.

The paper was about half that length and contradicts itself in several places. Hard for something that contradicts itself to be more than half true.

He didn’t have the foresight to avoid collapsing his hedge fund in a minor selloff by hedging.

You might be overselling his ability to predict the future.

Is "valedictorian" a good signal? I'm from Europe so we don't have the concept, but it seems to me that being the exact single topmost student in a cohort would set a perverse incentive to game the rules and hyperfocus on getting this label. To me being somewhere in the top 5%-10% while ignoring some bullshit and not caring about maximizing all grades would indicate a more robust mind that is focused on better priorities. (I'm not claiming it is easy to become valedictorian, I'm just wondering what precisely it indicates about a person, besides obvious talent and conscientiousness etc)

I'm not sure failing to understand the concept of hedging and the difference between mean and median returns under heavily leveraged trading counts as generational foresight. Most 25 year olds I know in the tech world have been calling for AGI since about the same time. I do respect his ability to just dive headfirst into something even though he basically took wallstreetbets literally but with billions of dollars under management rather than thousands. I'm not sure why we revere having enough confidence to do something badly so much.

Early 2024 is a bit late, isn't it? He jumped on a bandwagon and his AGI fever ramblings quoted in the submission show that he is just a marketing drone.

You sure that's the right number of nines? We'd probably need to look at over a billion earths to find another such 25 year old in that case. Further, if we look at the rest of his body of work, it may temper our estimation of his foresight.

You forgot to add that he was in a relationship and later engaged to the chief of staff at Anthropic while raising and running his fund, but I’m sure he had no access to material non-public information.

Do you really need a 300 page thesis to say in 2024 that AI is going to become a big deal?

My 15 year old said the same things in a few paragraphs. But then, he doesn't have the friends from having worked with Sam Bankman-Fried.

Well that “foresight” left him to walk right into a total amateur hour cash crunch that would have gotten a junior Wall Street analyst fired… so the evidence of amazing foresight here isn’t great.

Calling the fund “situational awareness” and then demonstrating a complete lack of situational awareness was rather amusing.

Looking at a kid in his 20s that says "AI will make infinite money" and then loses all his money:

> It's generational foresight not available to 99.999999999999999999% of 25 year olds.

> not available to 99.999999999999999999%

There are not enough human 25 years old in history for this many 9s. You had 21 nines, which would only make sense if there were 10 billion billion humans ever and that he is the only one. However, the current estimate is there were 107 billion humans ever alive in history, so your statement is impossible even if the estimate is off by 10000 fold.

I don't understand how you can read a thesis that predicts AGI in 2027 (and the subsequent explosion to ASI) and say that it "all ended up becoming true in 2026".

However, I think this is a classic example of the category error the OP is talking about. Making accurate predictions isn't necessarily the same thing as asset management. Once you reach that level of fame, people will 100% target and trade against you, and when someone is that well-known, their market positions inevitably get exposed—how could they possibly handle that?

> generational foresight

His strategy was to trade on MNPI available to him via relationships that no larger fund/bank compliance department would sign off on, in a portfolio no fund/bank risk department would allow.

> he wrote a 300 page thesis on AI in 2024 that all ended up becoming true in 2026

...no? The essay is literally titled "The Decade Ahead", its predictions have not come true in 2026.

> The AGI race has begun. We are building machines that can think and reason. By 2025/26, these machines will outpace many college graduates. By the end of the decade, they will be smarter than you or I; we will have superintelligence, in the true sense of the word

I'd argue even the 2025/26 part of that isn't true, though the "many" in "many college graduates" makes the claim very difficult to verify.

But even putting all that aside I think it's still a valuable evidence point when discussing "intellectual arrogance". No doubt this 25 year old is very smart when it comes to AI. But there's this belief that if someone is smart at one thing then they're surely smart at everything. Turns out "smart at AI" doesn't automatically translate to "can sensibly manage a $45bn fund".

A lot of sci-fi authors have published in their 20s.

You missed the years long work at FTX under Sam Bank Friedman all the way to FTX's bankruptcy.

Not sure if that's true. Tons of r/wallstreetbets redditors did the same.

> 99.999999999999999999% of 25 year olds

18 decimal places... goddamn that's more than you get on a computer usually. We must be counting 25 year olds that won't even be born for millenia.

Appreciate the context

Returns? So soon in the lifecycle of a VC fund?

This holds for all good performance art. I suck at writing in general and I'd suck even more at writing the scenario and support materials for performance art.

(I mean sure from sibling comments I get that this is, in fact, not performance art but you gotta admit that if it were it'd be damn good!!!)

Make sure to put that front and center on the website next time.

Not FTX the exchange, he was working at the FTX Future Fund which is part of the philanthropic foundation SBF created as part of his effective altruism commitment (the same SBF who also provided $500M to help Anthropic get started btw, just a few weeks before going bust).

As a side note, it’s pretty insane how many people of the AI industry have a relationship to effective altruism

Why is anyone from FTX being trusted with money…

The word you're looking for is hubris and it has been endemic amongst some groups of software engineers for 25 years.

He’s talking about safety guardrails where the model shuts off.

anti-ai-bots? people who dislike LLM et al use LLM to repeat what is public opinion anyway? like, where?

Right, we have flags. And this post has been flagged off the front page of HN even with 150 upvotes in less than 2 hours...

So HN isn't included in the "wherever it gets posted on the internet" since it's implied that "wherever" needs downvote capabilities. Doesn't make the sentence any less true - you just read too much into it.

I was just venting (ironic, I know).

May I ask, did you get anything out of it? I felt I wasted my time reading it, but that's what makes me think I'm missing something.

Sorry, I was just venting.