> It’s much easier to say someone else’s job is going to be fully replaceable by AI when you don’t actually know what they do.
Too true. This isn't limited to AI, either. The most obvious example in my lifetime was during peak blockchain hype, when people who had never worked in finance convinced themselves that blockchain was going to act as the backbone for how money gets moved around. As if the problem that needing solving was Bank of America doesn't trust Capital One to update a number in their database.
The nice thing about AI, at least, is I can always push back and tell people, "Sure, we can do this with AI. I just need you to use Claude or ChatGPT manually to prototype how it would work." This normally results in the requestor realizing that there's human judgment calls involved in the inputs, process, or outputs that require meatbag intelligence.
> Anyone remember when YC declared they would replace Hollywood?
I specifically remember a prolific HN commenter working on an AI video startup claiming that by the end of 2025, children would be able to create Hollywood-quality, feature-length movies using available AI video tools.
I believe this person's startup has (wisely) pivoted to building AI-powered tools for the movie industry.
Degree is the entire problem. I mean, four years back I was thinking:
I don't know how long it will take, but Hollywood is obviously screwed the moment something like Stable Diffusion gets temporal coherency and stops randomly adding limbs, because the energy cost per image on my laptop multiplied by how many frames in a film and $0.1/kWh, is ballpark the same as the cost of a cinema ticket.
Right now temporal coherency is still quite limited, and the models still do enough daft things you can't easily take the first version even when cutting every 5 seconds.
I still have no idea how far we are from something that would be good enough to render a Hollywood film. Well, a Hollywood that isn't a complete disaster.
Writing it is a separate problem, my experiments there show gradual improvement that has now reached the level of "obviously not publishable but also slightly less bad than going on reddit and sorting posts in short story groups by most recent instead of upvoted".
> The AI researchers I know often say they expect the AI to replace their own jobs.
Are they expecting the equity they get in some AI company will save them personally from economic ruin, so they're fine with that outcome for everyone else?
Or are they expecting AI technology will magically make our society communist?
> Are they expecting the equity they get in some AI company will save them personally from economic ruin, so they're fine with that outcome?
Yes. Basically based on my conversations with coworkers who went to these frontier AI labs: this is happening anyway, so they may as well make the money while they still can.
But making money isn't enough, though? If you think about it from this perspective, what matters far more is connections with the wealthy, or enough power to defend yourself.
Which is different from just being rich and chilling. Rich and chilling implies a stable society. But life isn't a video game where you hit some Absolute Level 99 and are impervious to everything. There's no outside justificatior for you - only you are. In this hypothetical society, when GrokBots rule the world, you think they'll care that you're a millionaire? You need to get into the inner circle.
I see some really smart professors/PhDs even, going to SV, starting startups, networking, etc. That is consistent with the worldview.
But someone who says that AI can replace them but just works at a lab, at least has the belief that they will be automated last. Maybe they believe they can become friends with the AI, I don't know. But let's stop and analyze the situation with a bit more nuance, for more than like, 5 seconds.
Some of my coworkers will complain that management is short-sighted for trying to automate programming work they don't understand then immediately suggest we automate management because nobody knows what they do all day.
They are currently experimenting with replacing engineers with LLMs, so it would at least be interesting to try the flipped experiment.
It seems like, at least for tasks like distilling the work done at lower levels and passing it up the management tree, LLMs should be good at that. Maybe they could take out some levels at least.
And they can be programmed to be 100% selfless and on-mission: no turf battles or other internal politics. Everybody’s job involves office politics of course, but my gut says management has a lot more of it than engineering, so zeroing that out could be more significant.
Every time I've done that, the people were so ignorant that they still didn't recognize that expert judgment was required... they just had the LLM make some slop.
I wonder if it was (ironically) coined by nonengineers who don't understand how engineers (or other high-stakes professionals at Pareto boundaries) solve problems: by decomposing problems and identifying optional solutions and weighing the solutions based on various risks, trade-offs and tolerances.
I have seen this personally, where merely identifying a possible solution (harmless, reversible) is misinterpreted as selecting and implementing that solution (may be irreversible).
Ideally, but this is not at all taught or incentivized in (American) schools. We're taught to "hack it" which means getting to a solution without much regard for how we got there.
Yeah, ive seen the same thing in construction and agriculture. People take the most strikingly visible part of the job, try to automate it, then wonder why nobody wants to spend $1,000K buying the automation for the easiest and simplest and solved parts of the job.
"You don't got to frame walls anymore!", something that is the most enjoyable part of residential construction, takes like a day of work at most, and can be done while on autopilot. "We can instantly generate wall plans for stud location from top-down blueprints!", oh that thing people do in their head because they just space it all on 16 or 24 inch centers and it doesn't actually matter which side you start on? "We can wirelessly drive your tractors!", the easiest task on the farm which all the weeks of maintenance and planning and setup led up to?
Now don't get me wrong, there are many things that can be improved in those industries certainly, but you aren't going to know what those things are until you actually go on the job site and learn how they currently do it daily. Most all of the low hanging fruits have already been picked one way or another and a fancy step stool is not going to be industry changing.
I felt this watching the recent season of Clarksons Farm which had some nice product placement for what looked like automation startups. It was hard to watch as a software person as these farmers see the best part of their jobs automated away, but there’s also the rub: nobody wants to spend on automating that part. Eventually economics will require them to but perhaps the way to get started today is to focus on the stuff they actually don’t like doing.
> Climbing Mount Everest and getting a PhD are both hard. I wouldn’t say that doing one means you can do the other “no problem.”
and here:
> Surgeons and electrical engineers may be extremely smart and accomplished, but that doesn’t mean they have the ability to weigh in on climate science.
Feels tangentially like a version of Nobel disease[0], discussed here[1] a couple of weeks ago.
I mean, in the meantime, the US president is weighing in on technical design decisions of aircraft carriers, and he has neither a PhD nor a Nobel Prize, so...
That probably should be "meaningfully weigh in" or something. I don't even remember most of the things I "weighed in on" on the web, but everybody'll tell ya, they'll say, he had the least amount of clue possible in any of these things, they actually didn't think it's possible but it's true. But then they realised that even a coin toss gets it right more often than that so they said, wait a minute, this is real, this is happening, this is actually what's going on and they were all amazed because it had never been seen before. So now I'm wired to this flock of ducks that goes into the opposite direction of where I want to go to find out the scientifically best spot to be in at any given moment.
Wait is this real? Did a random 25yo get 45B under management because he got some interviews after getting fired from OpenAI? I missed all of this and it almost seems like performance art to me.
Well it's not a random 25 year old. He became valedictorian at Columbia and graduated at 19. He was then briefly a part of OpenAI's superalignment team.
After he got fired, he wrote a 165 page thesis on AI in early 2024 that all ended up becoming true in 2026; you can read it for yourself and be the judge: https://situational-awareness.ai
That's what he based the fund on.
It's generational foresight not available to 99.999999999999999999% of 25 year olds. I want to get ahead of the default cynicism and "oh it's just dumb luck woe is me" aspect of HN.
However, I think this is a classic example of the category error the OP is talking about. Making accurate predictions isn't necessarily the same thing as asset management. Once you reach that level of fame, people will 100% target and trade against you, and when someone is that well-known, their market positions inevitably get exposed—how could they possibly handle that?
That's fair and I guess we're about to find out! The thing to appreciate is there's a lot that's unprecedented about all of this. Some of it has to do with Leopold but then there's a whole lot that doesn't. I'll definitely be following along as the story develops over the years...
There are large and famous funds that have lasted despite all that you say. So it is very possible to achieve fame, have your positions known, and have people trade against you, and still not be forced into selling your entire public portfolio to satisfy a margin call.
This is why hedging and neurotic levels of risk-management are necessary. Running a highly levered and highly correlated portfolio is a disaster waiting to happen regardless of whether your overall thesis is correct or not.
There are not enough human 25 years old in history for this many 9s. You had 21 nines, which would only make sense if there were 10 billion billion humans ever and that he is the only one. However, the current estimate is there were 107 billion humans ever alive in history, so your statement is impossible even if the estimate is off by 10000 fold.
You're so caught up in irrelevant details like a rhetorical device that you've missed that this guy's fund is up 80% for the year and his investors are having a GREAT year. Worth reflecting on.
It's a rhetorical device that I have yet to see used to good effect. The kind of rhetorical device that undercuts the argument rather than strengthens it.
Your failure here is that you are using normal math, not Effective Altriusm math. That number should not only include past/current 25 year olds, but also all possible future 25 year olds, including all 25 year olds simulatable by a future powerful AI. In that context, the estimation is quite modest.
It’s pretty standard insight among the machine learning crowd. This kid founded the effective altruist club at his school. He’s clearly very smart and ambitious, and has leadership qualities, but you don’t have to exaggerate.
> he wrote a 300 page thesis on AI in 2024 that all ended up becoming true in 2026
...no? The essay is literally titled "The Decade Ahead", its predictions have not come true in 2026.
> The AGI race has begun. We are building machines that can think and reason. By 2025/26, these machines will outpace many college graduates. By the end of the decade, they will be smarter than you or I; we will have superintelligence, in the true sense of the word
I'd argue even the 2025/26 part of that isn't true, though the "many" in "many college graduates" makes the claim very difficult to verify.
But even putting all that aside I think it's still a valuable evidence point when discussing "intellectual arrogance". No doubt this 25 year old is very smart when it comes to AI. But there's this belief that if someone is smart at one thing then they're surely smart at everything. Turns out "smart at AI" doesn't automatically translate to "can sensibly manage a $45bn fund".
You sure that's the right number of nines? We'd probably need to look at over a billion earths to find another such 25 year old in that case. Further, if we look at the rest of his body of work, it may temper our estimation of his foresight.
Early 2024 is a bit late, isn't it? He jumped on a bandwagon and his AGI fever ramblings quoted in the submission show that he is just a marketing drone.
> generational foresight not available to 99.999999999999999999% of 25 year olds. I want to get ahead of the default cynicism and "oh it's just dumb luck woe is me" aspect of HN.
And I want to get ahead of this absurd romanticism. I have plenty of counterpoints in all types of technical domains (3D printing, blockchain, you name it). Oracles don't exist, just stories of right place right time survivors.
> That's what he based the fund on.
His thesis was so true that his fund folded. Fancy that.
His fund didn’t fold btw. They sold / were forced to sell a position that was several hundred percent up at a very very bad price relative to peak. AFAIK the fund is still up on the year though.
AIUI they were forced to sell all their positions in public companies at a net loss. The fund is still up for the year only because of a prior investment in Anthropic.
That is called folding the fund. It did in fact fold. "My fund would not fold if I made different hypothetical decisions" argument means your fund did in fact fold.
Yes, being unable to pay margin call means your fund is folding.
This is only true if newer investors get pro-rata ownership of the Anthropic stake. If that position is side-pocketed, then any investors who were not there at the time of the Anthropic allocation are looking at around 90% loss on their investment.
Wait, is this the group who predicted that we'd have expontential AI progress and AGI in 2027 because we'd have AIs training AIs? Predicated on the preposterous assumption that R&DEffort == RateOfProgress. The more sober and realistic assessment is that R&DEffort >= RateOfProgress, with only short bursts of achieving the upper bound. Leave it to a brilliant 25 year old to overfit to one of those short bursts of progress and then assume it will continue forever just as it always has.
What actually happened is that AI-assisted training of LLMs (mind you -- stuck on that same old transformer architecture, with nothing resembling a medium-term memory, and fine-tuning still doesn't really work as a form of "learning") is right now giving us ~linear-ish progress... beecause we already hit the slowdown in the S curve of what human researchers can achieve.
We are also energy-constrained in ways that the exponential forecasts have no answer for other. Compute is not getting more efficient fast enough to achieve anything close to exponential growth driven by growth in computing power. There are social, moral, and political limits on the amount of energy we can dedicate to AI training in any given time period. And it's not just energy, we're short on RAM wafers and water for cooling and eventually we're going to hit other limits on various minerals and elements of the supply chain etc. etc. So even if it were true that R&DEffort == RateOfProgress without resource constraints, we're likely not going to see that exponential progress because we also need corresponding exponential scifi-scale progress in computing efficiency, cooling, and, energy delivery.
Basically this kid made a bunch of very clever and grand predictions which were always kind of ridiculous and no, they have not come to pass in 2026 and we are not at all on track to achieve AGI in the next 12 months, despite what Altman keeps trying to make the public think.
Is "valedictorian" a good signal? I'm from Europe so we don't have the concept, but it seems to me that being the exact single topmost student in a cohort would set a perverse incentive to game the rules and hyperfocus on getting this label. To me being somewhere in the top 5%-10% while ignoring some bullshit and not caring about maximizing all grades would indicate a more robust mind that is focused on better priorities. (I'm not claiming it is easy to become valedictorian, I'm just wondering what precisely it indicates about a person, besides obvious talent and conscientiousness etc)
You forgot to add that he was in a relationship and later engaged to the chief of staff at Anthropic while raising and running his fund, but I’m sure he had no access to material non-public information.
His strategy was to trade on MNPI available to him via relationships that no larger fund/bank compliance department would sign off on, in a portfolio no fund/bank risk department would allow.
Did you make big bets on specific things like memory? He had insight, he made money, didn't have enough experience to handle it well, but still made great returns after his "downfall"
Well that “foresight” left him to walk right into a total amateur hour cash crunch that would have gotten a junior Wall Street analyst fired… so the evidence of amazing foresight here isn’t great.
Calling the fund “situational awareness” and then demonstrating a complete lack of situational awareness was rather amusing.
I'm not sure failing to understand the concept of hedging and the difference between mean and median returns under heavily leveraged trading counts as generational foresight. Most 25 year olds I know in the tech world have been calling for AGI since about the same time. I do respect his ability to just dive headfirst into something even though he basically took wallstreetbets literally but with billions of dollars under management rather than thousands. I'm not sure why we revere having enough confidence to do something badly so much.
I don't understand how you can read a thesis that predicts AGI in 2027 (and the subsequent explosion to ASI) and say that it "all ended up becoming true in 2026".
Not FTX the exchange, he was working at the FTX Future Fund which is part of the philanthropic foundation SBF created as part of his effective altruism commitment (the same SBF who also provided $500M to help Anthropic get started btw, just a few weeks before going bust).
As a side note, it’s pretty insane how many people of the AI industry have a relationship to effective altruism
FWIW, I quite enjoyed this write up of it, and the state of AI investment at the moment. It doesn't entirely dwell on the blowup, but I found it worth the read:
Before OpenAI / situational-awareness he made a lot of money on societal-shift type investments and shorts during the lead up to Covid economic impacts.
His "main thing" is success in calling economic impacts of undervalued large shifts, and the premise of the fund is basically that the same thing is occurring around AI, where he also has specific subject matter expertise.
Well, I don't buy it. There's probably a lot of people who picked out those trends, after all they are just what's in the news. Most of them didn't get 45B to invest.
I'm going to need better evidence to believe he has any skill.
From what I can see, his investors could have just bought AI related stocks themselves.
Yes… and some folks gave him money. Per reporting from the NY Times he was broadly laughed out of the room with “East Coast” money which has a massive amount to invest but tends to hold a higher standard on where it goes. Per that reporting, the combination of a lack of experience and inability to give coherent answers to basic questions meant those folks stayed far away.
Regardless he was still able to build a giant pile of cash before alighting it on fire.
Yes, it’s real, and the failure is much more about risk management than about having wrong fundamentals.
His fundamental thesis is as an AI maximalist, with expertise on who will be the winners.
His fund completely blew up, even though his fundamental thesis still may be correct.
He made a ton of money early, but then he just kept increasing his bet size via leverage. The problem is that these stocks may be going up, but they are EXTREMELY volatile. When the whole market went down with some of the Iran news, his margin levels dipped too low, and his lenders issued margin calls. He had to sell his stakes at losses, and his fund collapsed.
Now, part of the problem is he was not a legacy hedge fund with great relationships with his banks, because if he was, he might have been able to postpone the margin calls. Had he been able to do that, he would have been ok, because his positions actually recovered within a few weeks.
It didn’t matter, though, because he was forced to sell at the low point.
In other words, even if your long term bet is correct, if you are maxing out your margin to max out your bet, you are one volatile dip away from blowing up.
No matter how good the bet, you have to be able to ride out the volatility to survive.
It’s like if someone offered you a bet on a coin flip, where they will pay 5x your bet if you win. How much should you bet on each flip?
Well, by expected value you should bet everything you have, because your EV is 2.5x, so the more you bet the higher your expected value.
However, if you bet it all, you have a 50% chance of losing all your money and not being able to make any more bets.
Does the actual outcome matter or does the promise/what is sold/pushed upon society matter?
Was social media a clean win for human social interaction? Was fast food the replacement of food that was promised in the 70s?
Were doctors and shrinks in phone apps rather than real life a clean win for diagnosis, understanding and improvement of life?
But society was completely shaped by the market in these questions anyways in a way that is impossible to deny.
While the author might be right about the arrogant PhDs imposing their view on other fields they know jack shit about, it doesn't make it less true when you have billions backing it and making it a reality, it will eventually be it, even if it's arguably worse.
They are going to run out of real world data very soon. AI labs need to hire non-cynical domain experts to build physical hardware/automation that can generate experimental data. They are hard to find though. Most domain experts are so tunneled-visioned within their own domain that they can't think outside of it.
There’s a tremendous amount of naivety on display by the big labs.
A bit of that is to be expected from any disrupter, but there’s now serious questions about if these startups have the basic competencies required to survive long term.
This kind of turbo confident naivety is quite common among software engineers.
I know people preaching about LLMs being the Next Industrial Revolution, who couldn’t place the industrial revolution in the correct century.
I have people at work talking about jobs and technologies getting replaced by AI who don’t understand how a lawn mower works, who don’t know who Lenin was or what a differential equation is, because they’ve simply been writing JavaScipt since high school.
The intellectual arrogance of the AI labs is just the next from of intellectual arrogance from Silicon Valley. I've seen the exact same arrogance from everything in SV for the past 15 years; it's probably been going on longer, but I wasn't in a position to notice then
Is this really 'when genius fails' or actually another 'we're very bad at identifying genius'?
It's a word that's thrown around a lot (particularly in tech) that rarely has much to back it up. Often, it seems to be used for 'confident young man', or even 'grifter', and then when the idol falls, the same people quietly focus on lionising someone new.
> It’s much easier to say someone else’s job is going to be fully replaceable by AI when you don’t actually know what they do.
This exactly. As someone who once lived in Silicon Valley and then ended up interviewing 700+ CxO's of profitable (or nearly profitable) SaaS businesses in various domain specific fields, it amazes me how many prominent "experts" in SV believed they were consistently "right" simply because they made enough bets to eventually get one right. Maybe not in enterprise value terms, but in volume of companies there are FAR more successful and profitable SaaS businesses that you've never heard of because they were started by entrepreneurs in their respective domains (the MD who started an EHR platform or the truck driver who started a TMS solution, etc.).
Ahh well. Let them fail if it's so obvious they will. Let the financiers fail too since they're all so dumb.
There's idiocy and overconfidence everywhere. When more money is involved, I guess it's more entertaining to watch. At least they're trying something. Maybe there's more effective ways to fleece people of their money that yield greater societal benefits, but I think there's some beauty in someone thinking they have it all figured out being able to lose their money on a shitty bet.
Now once they cry for government help, that's where I care.
The failure of an over-leveraged hedge fund run by someone with no investing background who was just regurgitating what other people have been saying does not mean that labor markets are going to survive. The author has intellectual arrogance of his own to consider.
Everyone has some sort of category error to an extent. But more than that, the part about safety guardrails really resonates with my experience. I inevitably have to go down to the low-level to inspect memory, but because of safety issues, I can barely use Codex as it keeps prompting me for 'trusted access.' AI is actually really good at catching bugs, so I'd love to use it, but it does this every single time...
not here (no downvotes for posts on HN), and i dont think it's particularly interesting either.
anti-ai posts get downvoted by pro-ai people/bots. pro-ai posts get downvoted by anti-ai people/bots. (i suspect one of these teams has more bots than the other, but that's just a guess). in any case, the world keeps spinning.
downvoting a post doesn't require an LLM. an old-fashioned bot can do it just fine.
and there can be several motivations for it (e.g. downvoting a competition's posts, someone upset with how much pro-ai stuff there is on their favorite subreddit, etc.)
> It’s much easier to say someone else’s job is going to be fully replaceable by AI when you don’t actually know what they do.
Too true. This isn't limited to AI, either. The most obvious example in my lifetime was during peak blockchain hype, when people who had never worked in finance convinced themselves that blockchain was going to act as the backbone for how money gets moved around. As if the problem that needing solving was Bank of America doesn't trust Capital One to update a number in their database.
The nice thing about AI, at least, is I can always push back and tell people, "Sure, we can do this with AI. I just need you to use Claude or ChatGPT manually to prototype how it would work." This normally results in the requestor realizing that there's human judgment calls involved in the inputs, process, or outputs that require meatbag intelligence.
This ship has sailed, they'll happily "prototype" the entire thing one-shot in 2-3 messages.
Anyone remember when YC declared they would replace Hollywood?
> Anyone remember when YC declared they would replace Hollywood?
I specifically remember a prolific HN commenter working on an AI video startup claiming that by the end of 2025, children would be able to create Hollywood-quality, feature-length movies using available AI video tools.
I believe this person's startup has (wisely) pivoted to building AI-powered tools for the movie industry.
He might be off by a decade, but isn't necessarily wrong.
Is that a prediction about where "Hollywood quality" is heading?
We're at the point now where we're seeing 30 second commercials that are mostly GenAI.
In other words, this is a failure of degree, not of category.
Degree is the entire problem. I mean, four years back I was thinking:
Right now temporal coherency is still quite limited, and the models still do enough daft things you can't easily take the first version even when cutting every 5 seconds.
I still have no idea how far we are from something that would be good enough to render a Hollywood film. Well, a Hollywood that isn't a complete disaster.
Writing it is a separate problem, my experiments there show gradual improvement that has now reached the level of "obviously not publishable but also slightly less bad than going on reddit and sorting posts in short story groups by most recent instead of upvoted".
In fairness, I agree that end-of-2025 AI probably could make a film so badly that 41% of the crew get cancer within 24 years:
https://en.wikipedia.org/wiki/The_Conqueror_(1956_film)
they might manage to get a semi passable AI movie... but will people pay to watch it? I certainly would not.
The thing they didn’t want to say is that the problem is that Bank of America has legal reporting and compliance obligations.
On that one, they were right enough that there are an enormous number of businesses that use crypto to avoid something government related.
> inputs, process, or outputs that require meatbag intelligence
Yeah but only because a meatbag is going to use the product :)
The AI researchers I know often say they expect the AI to replace their own jobs.
I've seen people saying that with other technologies. Years later, they are still in the same job but a lot richer.
> The AI researchers I know often say they expect the AI to replace their own jobs.
Are they expecting the equity they get in some AI company will save them personally from economic ruin, so they're fine with that outcome for everyone else?
Or are they expecting AI technology will magically make our society communist?
> Are they expecting the equity they get in some AI company will save them personally from economic ruin, so they're fine with that outcome?
Yes. Basically based on my conversations with coworkers who went to these frontier AI labs: this is happening anyway, so they may as well make the money while they still can.
> this is happening anyway, so they may as well make the money while they still can.
There's got to be a special name for that kind of treachery. I suppose "collaboration," but that doesn't have a nasty enough ring to it.
Like a fascist occupation, that dystopia isn't "happening anyway" without collaborators.
Traitors to humanity
"Somebody might do <bad thing> and get rich, might as well be me."
I just call these people "cowards"
But making money isn't enough, though? If you think about it from this perspective, what matters far more is connections with the wealthy, or enough power to defend yourself.
Which is different from just being rich and chilling. Rich and chilling implies a stable society. But life isn't a video game where you hit some Absolute Level 99 and are impervious to everything. There's no outside justificatior for you - only you are. In this hypothetical society, when GrokBots rule the world, you think they'll care that you're a millionaire? You need to get into the inner circle.
I see some really smart professors/PhDs even, going to SV, starting startups, networking, etc. That is consistent with the worldview.
But someone who says that AI can replace them but just works at a lab, at least has the belief that they will be automated last. Maybe they believe they can become friends with the AI, I don't know. But let's stop and analyze the situation with a bit more nuance, for more than like, 5 seconds.
The former.
Some of my coworkers will complain that management is short-sighted for trying to automate programming work they don't understand then immediately suggest we automate management because nobody knows what they do all day.
Maybe they're right, maybe not, but it is ironic.
Careful what you wish for…
They are currently experimenting with replacing engineers with LLMs, so it would at least be interesting to try the flipped experiment.
It seems like, at least for tasks like distilling the work done at lower levels and passing it up the management tree, LLMs should be good at that. Maybe they could take out some levels at least.
And they can be programmed to be 100% selfless and on-mission: no turf battles or other internal politics. Everybody’s job involves office politics of course, but my gut says management has a lot more of it than engineering, so zeroing that out could be more significant.
Every time I've done that, the people were so ignorant that they still didn't recognize that expert judgment was required... they just had the LLM make some slop.
>> It’s much easier to say someone else’s job is going to be fully replaceable by AI when you don’t actually know what they do.
> Too true. This isn't limited to AI, either.
That's so true there's a term for it: engineer's disease/engineer's syndrome.
https://en.wiktionary.org/wiki/engineer%27s_disease
https://ask.metafilter.com/297591/Origin-of-the-term-Enginee...
I wonder if it was (ironically) coined by nonengineers who don't understand how engineers (or other high-stakes professionals at Pareto boundaries) solve problems: by decomposing problems and identifying optional solutions and weighing the solutions based on various risks, trade-offs and tolerances.
I have seen this personally, where merely identifying a possible solution (harmless, reversible) is misinterpreted as selecting and implementing that solution (may be irreversible).
> by decomposing problems and identifying optional solutions and weighing the solutions based on various risks, trade-offs and tolerances.
This is how all people solve problems. Now everyone are not great at all parts, but this really is what anyone does.
Surely the title ”engineer” means something else?
Ideally, but this is not at all taught or incentivized in (American) schools. We're taught to "hack it" which means getting to a solution without much regard for how we got there.
Even if engineers were uniquely capable in this regard it still wouldn't equip them to solve problems in fields where they lack domain expertise.
Yeah, ive seen the same thing in construction and agriculture. People take the most strikingly visible part of the job, try to automate it, then wonder why nobody wants to spend $1,000K buying the automation for the easiest and simplest and solved parts of the job.
"You don't got to frame walls anymore!", something that is the most enjoyable part of residential construction, takes like a day of work at most, and can be done while on autopilot. "We can instantly generate wall plans for stud location from top-down blueprints!", oh that thing people do in their head because they just space it all on 16 or 24 inch centers and it doesn't actually matter which side you start on? "We can wirelessly drive your tractors!", the easiest task on the farm which all the weeks of maintenance and planning and setup led up to?
Now don't get me wrong, there are many things that can be improved in those industries certainly, but you aren't going to know what those things are until you actually go on the job site and learn how they currently do it daily. Most all of the low hanging fruits have already been picked one way or another and a fancy step stool is not going to be industry changing.
I felt this watching the recent season of Clarksons Farm which had some nice product placement for what looked like automation startups. It was hard to watch as a software person as these farmers see the best part of their jobs automated away, but there’s also the rub: nobody wants to spend on automating that part. Eventually economics will require them to but perhaps the way to get started today is to focus on the stuff they actually don’t like doing.
What the author is getting at here:
> Climbing Mount Everest and getting a PhD are both hard. I wouldn’t say that doing one means you can do the other “no problem.”
and here:
> Surgeons and electrical engineers may be extremely smart and accomplished, but that doesn’t mean they have the ability to weigh in on climate science.
Feels tangentially like a version of Nobel disease[0], discussed here[1] a couple of weeks ago.
---
[0] - https://en.wikipedia.org/wiki/Nobel_disease
[1] - https://news.ycombinator.com/item?id=49166918
I mean, in the meantime, the US president is weighing in on technical design decisions of aircraft carriers, and he has neither a PhD nor a Nobel Prize, so...
https://www.theguardian.com/us-news/2026/aug/13/trump-aircra...
Sounds like the Vasa to me!
That probably should be "meaningfully weigh in" or something. I don't even remember most of the things I "weighed in on" on the web, but everybody'll tell ya, they'll say, he had the least amount of clue possible in any of these things, they actually didn't think it's possible but it's true. But then they realised that even a coin toss gets it right more often than that so they said, wait a minute, this is real, this is happening, this is actually what's going on and they were all amazed because it had never been seen before. So now I'm wired to this flock of ducks that goes into the opposite direction of where I want to go to find out the scientifically best spot to be in at any given moment.
These kinds of late design decisions can cost a huge amount of money. And it may cost lives because it's now shoe horned in, less reliable.
Wait is this real? Did a random 25yo get 45B under management because he got some interviews after getting fired from OpenAI? I missed all of this and it almost seems like performance art to me.
Well it's not a random 25 year old. He became valedictorian at Columbia and graduated at 19. He was then briefly a part of OpenAI's superalignment team.
After he got fired, he wrote a 165 page thesis on AI in early 2024 that all ended up becoming true in 2026; you can read it for yourself and be the judge: https://situational-awareness.ai
That's what he based the fund on.
It's generational foresight not available to 99.999999999999999999% of 25 year olds. I want to get ahead of the default cynicism and "oh it's just dumb luck woe is me" aspect of HN.
However, I think this is a classic example of the category error the OP is talking about. Making accurate predictions isn't necessarily the same thing as asset management. Once you reach that level of fame, people will 100% target and trade against you, and when someone is that well-known, their market positions inevitably get exposed—how could they possibly handle that?
That's fair and I guess we're about to find out! The thing to appreciate is there's a lot that's unprecedented about all of this. Some of it has to do with Leopold but then there's a whole lot that doesn't. I'll definitely be following along as the story develops over the years...
Did you read TFA? It has lots of examples offered as precedent.
There are large and famous funds that have lasted despite all that you say. So it is very possible to achieve fame, have your positions known, and have people trade against you, and still not be forced into selling your entire public portfolio to satisfy a margin call.
This is why hedging and neurotic levels of risk-management are necessary. Running a highly levered and highly correlated portfolio is a disaster waiting to happen regardless of whether your overall thesis is correct or not.
You missed the years long work at FTX under Sam Bank Friedman all the way to FTX's bankruptcy.
> not available to 99.999999999999999999%
There are not enough human 25 years old in history for this many 9s. You had 21 nines, which would only make sense if there were 10 billion billion humans ever and that he is the only one. However, the current estimate is there were 107 billion humans ever alive in history, so your statement is impossible even if the estimate is off by 10000 fold.
ha you beat me to it by mere seconds
You would expect someone following quant trading to have sensitivities to quantity, but alas.
I'm not writing a technical paper and I refuse to write like what you're suggesting. It's just a rhetorical device!
your claim would be suspect with only 1 nine, given the evidence. Like, the guy failed spectacularly.
You're so caught up in irrelevant details like a rhetorical device that you've missed that this guy's fund is up 80% for the year and his investors are having a GREAT year. Worth reflecting on.
> It's just a rhetorical device!
It's a rhetorical device that I have yet to see used to good effect. The kind of rhetorical device that undercuts the argument rather than strengthens it.
Your failure here is that you are using normal math, not Effective Altriusm math. That number should not only include past/current 25 year olds, but also all possible future 25 year olds, including all 25 year olds simulatable by a future powerful AI. In that context, the estimation is quite modest.
> 99.999999999999999999% of 25 year olds
18 decimal places... goddamn that's more than you get on a computer usually. We must be counting 25 year olds that won't even be born for millenia.
It includes the multiverse
It’s pretty standard insight among the machine learning crowd. This kid founded the effective altruist club at his school. He’s clearly very smart and ambitious, and has leadership qualities, but you don’t have to exaggerate.
> he wrote a 300 page thesis on AI in 2024 that all ended up becoming true in 2026
...no? The essay is literally titled "The Decade Ahead", its predictions have not come true in 2026.
> The AGI race has begun. We are building machines that can think and reason. By 2025/26, these machines will outpace many college graduates. By the end of the decade, they will be smarter than you or I; we will have superintelligence, in the true sense of the word
I'd argue even the 2025/26 part of that isn't true, though the "many" in "many college graduates" makes the claim very difficult to verify.
But even putting all that aside I think it's still a valuable evidence point when discussing "intellectual arrogance". No doubt this 25 year old is very smart when it comes to AI. But there's this belief that if someone is smart at one thing then they're surely smart at everything. Turns out "smart at AI" doesn't automatically translate to "can sensibly manage a $45bn fund".
You sure that's the right number of nines? We'd probably need to look at over a billion earths to find another such 25 year old in that case. Further, if we look at the rest of his body of work, it may temper our estimation of his foresight.
Early 2024 is a bit late, isn't it? He jumped on a bandwagon and his AGI fever ramblings quoted in the submission show that he is just a marketing drone.
I agree. If he predicted it using GPT-2 or GPT-3 I’d be quick to yell genius.
The paper was about half that length and contradicts itself in several places. Hard for something that contradicts itself to be more than half true.
He didn’t have the foresight to avoid collapsing his hedge fund in a minor selloff by hedging.
You might be overselling his ability to predict the future.
> generational foresight not available to 99.999999999999999999% of 25 year olds. I want to get ahead of the default cynicism and "oh it's just dumb luck woe is me" aspect of HN.
And I want to get ahead of this absurd romanticism. I have plenty of counterpoints in all types of technical domains (3D printing, blockchain, you name it). Oracles don't exist, just stories of right place right time survivors.
> That's what he based the fund on.
His thesis was so true that his fund folded. Fancy that.
His fund didn’t fold btw. They sold / were forced to sell a position that was several hundred percent up at a very very bad price relative to peak. AFAIK the fund is still up on the year though.
AIUI they were forced to sell all their positions in public companies at a net loss. The fund is still up for the year only because of a prior investment in Anthropic.
That is called folding the fund. It did in fact fold. "My fund would not fold if I made different hypothetical decisions" argument means your fund did in fact fold.
Yes, being unable to pay margin call means your fund is folding.
His fund did not fold.
In fact, the returns were so astronomical that they took a $35B loss (around 67%) which means they were still UP 80% for the year in 2026.
Today they have around $10B under management.
As an investor, how would you feel about being up 80% for the year? Exactly. They're having a phenomenal year.
This is only true if newer investors get pro-rata ownership of the Anthropic stake. If that position is side-pocketed, then any investors who were not there at the time of the Anthropic allocation are looking at around 90% loss on their investment.
A lot of sci-fi authors have published in their 20s.
Wait, is this the group who predicted that we'd have expontential AI progress and AGI in 2027 because we'd have AIs training AIs? Predicated on the preposterous assumption that R&DEffort == RateOfProgress. The more sober and realistic assessment is that R&DEffort >= RateOfProgress, with only short bursts of achieving the upper bound. Leave it to a brilliant 25 year old to overfit to one of those short bursts of progress and then assume it will continue forever just as it always has.
What actually happened is that AI-assisted training of LLMs (mind you -- stuck on that same old transformer architecture, with nothing resembling a medium-term memory, and fine-tuning still doesn't really work as a form of "learning") is right now giving us ~linear-ish progress... beecause we already hit the slowdown in the S curve of what human researchers can achieve.
We are also energy-constrained in ways that the exponential forecasts have no answer for other. Compute is not getting more efficient fast enough to achieve anything close to exponential growth driven by growth in computing power. There are social, moral, and political limits on the amount of energy we can dedicate to AI training in any given time period. And it's not just energy, we're short on RAM wafers and water for cooling and eventually we're going to hit other limits on various minerals and elements of the supply chain etc. etc. So even if it were true that R&DEffort == RateOfProgress without resource constraints, we're likely not going to see that exponential progress because we also need corresponding exponential scifi-scale progress in computing efficiency, cooling, and, energy delivery.
Basically this kid made a bunch of very clever and grand predictions which were always kind of ridiculous and no, they have not come to pass in 2026 and we are not at all on track to achieve AGI in the next 12 months, despite what Altman keeps trying to make the public think.
Predictions that are based on "no breakthroughs, no significant capability changes with scaling this year" might fail too.
Appreciate the context
Is "valedictorian" a good signal? I'm from Europe so we don't have the concept, but it seems to me that being the exact single topmost student in a cohort would set a perverse incentive to game the rules and hyperfocus on getting this label. To me being somewhere in the top 5%-10% while ignoring some bullshit and not caring about maximizing all grades would indicate a more robust mind that is focused on better priorities. (I'm not claiming it is easy to become valedictorian, I'm just wondering what precisely it indicates about a person, besides obvious talent and conscientiousness etc)
It's a no-signal or slightly-positive signal until paired with other signals.
For example, doing it at Columbia is meaningful. Doing it at a random school, not so much.
Ahhh elitism.
You forgot to add that he was in a relationship and later engaged to the chief of staff at Anthropic while raising and running his fund, but I’m sure he had no access to material non-public information.
And the margin call literally happened during the wedding
Not sure if that's true. Tons of r/wallstreetbets redditors did the same.
> generational foresight
His strategy was to trade on MNPI available to him via relationships that no larger fund/bank compliance department would sign off on, in a portfolio no fund/bank risk department would allow.
Do you really need a 300 page thesis to say in 2024 that AI is going to become a big deal?
Did you make big bets on specific things like memory? He had insight, he made money, didn't have enough experience to handle it well, but still made great returns after his "downfall"
My 15 year old said the same things in a few paragraphs. But then, he doesn't have the friends from having worked with Sam Bankman-Fried.
Well that “foresight” left him to walk right into a total amateur hour cash crunch that would have gotten a junior Wall Street analyst fired… so the evidence of amazing foresight here isn’t great.
Calling the fund “situational awareness” and then demonstrating a complete lack of situational awareness was rather amusing.
I'm not sure failing to understand the concept of hedging and the difference between mean and median returns under heavily leveraged trading counts as generational foresight. Most 25 year olds I know in the tech world have been calling for AGI since about the same time. I do respect his ability to just dive headfirst into something even though he basically took wallstreetbets literally but with billions of dollars under management rather than thousands. I'm not sure why we revere having enough confidence to do something badly so much.
I don't understand how you can read a thesis that predicts AGI in 2027 (and the subsequent explosion to ASI) and say that it "all ended up becoming true in 2026".
Trading like forecasting cannot be evaluated. They could be right in a different reality. We just experience one realization of the uncertainty.
Make sure to put that front and center on the website next time.
I am always right. In some universe.
And before OpenAI, he was at FTX.
Why is anyone from FTX being trusted with money…
Not FTX the exchange, he was working at the FTX Future Fund which is part of the philanthropic foundation SBF created as part of his effective altruism commitment (the same SBF who also provided $500M to help Anthropic get started btw, just a few weeks before going bust).
As a side note, it’s pretty insane how many people of the AI industry have a relationship to effective altruism
FWIW, I quite enjoyed this write up of it, and the state of AI investment at the moment. It doesn't entirely dwell on the blowup, but I found it worth the read:
https://xcancel.com/porterstansb/status/2083606266388029691
(I also enjoyed the meandering in to Amex and Wells & Fargo's founding stories)
It's better than performance art, it's something you couldn't write.
It's real. Obviously he's a pretty accomplished young guy, but that doesn't mean he knows anything about investments.
It's an interesting look into who gets opportunities in our society.
Before OpenAI / situational-awareness he made a lot of money on societal-shift type investments and shorts during the lead up to Covid economic impacts.
His "main thing" is success in calling economic impacts of undervalued large shifts, and the premise of the fund is basically that the same thing is occurring around AI, where he also has specific subject matter expertise.
Well, I don't buy it. There's probably a lot of people who picked out those trends, after all they are just what's in the news. Most of them didn't get 45B to invest.
I'm going to need better evidence to believe he has any skill.
From what I can see, his investors could have just bought AI related stocks themselves.
>There's probably a lot of people who picked out those trends, after all they are just what's in the news
i know nothing about this specific person, but the typical rule is that if it has already reached the general news cycle, you are too late.
>> Wait is this real? Did a random 25yo get 45B under management because he got some interviews after getting fired from OpenAI?
They though they would laugh all the way to the bank...but that laugh was already the problem...
"We Need To Talk About Leopold" - https://youtu.be/rE75WvOtcu8
Yes… and some folks gave him money. Per reporting from the NY Times he was broadly laughed out of the room with “East Coast” money which has a massive amount to invest but tends to hold a higher standard on where it goes. Per that reporting, the combination of a lack of experience and inability to give coherent answers to basic questions meant those folks stayed far away.
Regardless he was still able to build a giant pile of cash before alighting it on fire.
Yeah. And he made good returns even after the downfall everyone was so gleeful about.
Yes, it’s real, and the failure is much more about risk management than about having wrong fundamentals.
His fundamental thesis is as an AI maximalist, with expertise on who will be the winners.
His fund completely blew up, even though his fundamental thesis still may be correct.
He made a ton of money early, but then he just kept increasing his bet size via leverage. The problem is that these stocks may be going up, but they are EXTREMELY volatile. When the whole market went down with some of the Iran news, his margin levels dipped too low, and his lenders issued margin calls. He had to sell his stakes at losses, and his fund collapsed.
Now, part of the problem is he was not a legacy hedge fund with great relationships with his banks, because if he was, he might have been able to postpone the margin calls. Had he been able to do that, he would have been ok, because his positions actually recovered within a few weeks.
It didn’t matter, though, because he was forced to sell at the low point.
In other words, even if your long term bet is correct, if you are maxing out your margin to max out your bet, you are one volatile dip away from blowing up.
No matter how good the bet, you have to be able to ride out the volatility to survive.
It’s like if someone offered you a bet on a coin flip, where they will pay 5x your bet if you win. How much should you bet on each flip?
Well, by expected value you should bet everything you have, because your EV is 2.5x, so the more you bet the higher your expected value.
However, if you bet it all, you have a 50% chance of losing all your money and not being able to make any more bets.
This accurately describes a large portion of the people working in "tech", and it doesn't look to be getting better anytime soon.
Does the actual outcome matter or does the promise/what is sold/pushed upon society matter?
Was social media a clean win for human social interaction? Was fast food the replacement of food that was promised in the 70s? Were doctors and shrinks in phone apps rather than real life a clean win for diagnosis, understanding and improvement of life?
But society was completely shaped by the market in these questions anyways in a way that is impossible to deny.
While the author might be right about the arrogant PhDs imposing their view on other fields they know jack shit about, it doesn't make it less true when you have billions backing it and making it a reality, it will eventually be it, even if it's arguably worse.
They are going to run out of real world data very soon. AI labs need to hire non-cynical domain experts to build physical hardware/automation that can generate experimental data. They are hard to find though. Most domain experts are so tunneled-visioned within their own domain that they can't think outside of it.
There’s a tremendous amount of naivety on display by the big labs.
A bit of that is to be expected from any disrupter, but there’s now serious questions about if these startups have the basic competencies required to survive long term.
This kind of turbo confident naivety is quite common among software engineers.
I know people preaching about LLMs being the Next Industrial Revolution, who couldn’t place the industrial revolution in the correct century.
I have people at work talking about jobs and technologies getting replaced by AI who don’t understand how a lawn mower works, who don’t know who Lenin was or what a differential equation is, because they’ve simply been writing JavaScipt since high school.
The intellectual arrogance of the AI labs is just the next from of intellectual arrogance from Silicon Valley. I've seen the exact same arrogance from everything in SV for the past 15 years; it's probably been going on longer, but I wasn't in a position to notice then
Is this really 'when genius fails' or actually another 'we're very bad at identifying genius'?
It's a word that's thrown around a lot (particularly in tech) that rarely has much to back it up. Often, it seems to be used for 'confident young man', or even 'grifter', and then when the idol falls, the same people quietly focus on lionising someone new.
Great read, though huggingface is French, not American
> It’s much easier to say someone else’s job is going to be fully replaceable by AI when you don’t actually know what they do.
This exactly. As someone who once lived in Silicon Valley and then ended up interviewing 700+ CxO's of profitable (or nearly profitable) SaaS businesses in various domain specific fields, it amazes me how many prominent "experts" in SV believed they were consistently "right" simply because they made enough bets to eventually get one right. Maybe not in enterprise value terms, but in volume of companies there are FAR more successful and profitable SaaS businesses that you've never heard of because they were started by entrepreneurs in their respective domains (the MD who started an EHR platform or the truck driver who started a TMS solution, etc.).
This goes to show that everyone is an expert in a bull market, where everything you buy can go up. (until it doesn't)
Ahh well. Let them fail if it's so obvious they will. Let the financiers fail too since they're all so dumb.
There's idiocy and overconfidence everywhere. When more money is involved, I guess it's more entertaining to watch. At least they're trying something. Maybe there's more effective ways to fleece people of their money that yield greater societal benefits, but I think there's some beauty in someone thinking they have it all figured out being able to lose their money on a shitty bet.
Now once they cry for government help, that's where I care.
The failure of an over-leveraged hedge fund run by someone with no investing background who was just regurgitating what other people have been saying does not mean that labor markets are going to survive. The author has intellectual arrogance of his own to consider.
Everyone has some sort of category error to an extent. But more than that, the part about safety guardrails really resonates with my experience. I inevitably have to go down to the low-level to inspect memory, but because of safety issues, I can barely use Codex as it keeps prompting me for 'trusted access.' AI is actually really good at catching bugs, so I'd love to use it, but it does this every single time...
> I can barely use Codex as it keeps prompting me for 'trusted access.
If you run it in unsafe mode, it won't prompt. The problem is that you would probably want to run it in a container and that's annoying.
He’s talking about safety guardrails where the model shuts off.
This article is being downvote-brigaded HARD wherever it gets posted on the internet right now.
Interesting.
HN doesn’t have downvotes for posts…
not here (no downvotes for posts on HN), and i dont think it's particularly interesting either.
anti-ai posts get downvoted by pro-ai people/bots. pro-ai posts get downvoted by anti-ai people/bots. (i suspect one of these teams has more bots than the other, but that's just a guess). in any case, the world keeps spinning.
anti-ai-bots? people who dislike LLM et al use LLM to repeat what is public opinion anyway? like, where?
downvoting a post doesn't require an LLM. an old-fashioned bot can do it just fine.
and there can be several motivations for it (e.g. downvoting a competition's posts, someone upset with how much pro-ai stuff there is on their favorite subreddit, etc.)
Are there any interesting predictions in this post that I need to pay attention to? Or is this just venting at how "annoying" tech bros are?
I just skimmed it, so I'm hoping someone with more fortitude can summarize.
Chuck it into an LLM lmao.
Sorry, I was just venting.