I still think it is interesting. This bubble (if you believe it) seems closer to tulips than it does to rail or real estate. People were investing in tulips with the possiblity that the market would completely crash, and tulips, without the frenzy behind them, are just tulips (not that useful). rails and real estate may loose value, but there was never any real possibilty that they would become valueless. GPU's are interesting in that they are like tulips (a breakthrough in efficiency could render them basically worthless), but they literally have the worthless aspect built into them directly. You will not be able to profitably run an LLM on a 5 year old GPU, as your competitors will be able to run inference at much higher efficiencies than you with modern chips and will undercut you on price.
It is just different because it is an almost guaranteed point in the future that they become worthless (unlike rail or real estate).
> GPU's are interesting in that they are like tulips (a breakthrough in efficiency could render them basically worthless), but they literally have the worthless aspect built into them directly.
Don't tulips have the "worthless aspect" AKA end-of-life built in as well? They perish, after all!
> You will not be able to profitably run an LLM on a 5 year old GPU, as your competitors will be able to run inference at much higher efficiencies than you with modern chips and will undercut you on price.
If you're in the business of selling tokens, certainly. If you're in the business of something else, and use LLMs to speed a process up, do you care that the 1-day process now takes 2 hours on 5yo hardware and 1 hour on SOTA hardware?
The only businesses in trouble here are those in the business of selling tokens. Those businesses selling potatoes who use an LLM to streamline/shorten some business process aren't going to care.
This comment reminds me of this paper which makes a similar argument. It claims that chips are more than just a depreciating asset you actually have to keep buying more for the economics to enable the scaling. And inherently there's going to be ups and downs. The AI labs seem to not be preparing for the depreciation nor the scaling requirements and that puts their economics into question.
I still think it is interesting. This bubble (if you believe it) seems closer to tulips than it does to rail or real estate. People were investing in tulips with the possiblity that the market would completely crash, and tulips, without the frenzy behind them, are just tulips (not that useful). rails and real estate may loose value, but there was never any real possibilty that they would become valueless. GPU's are interesting in that they are like tulips (a breakthrough in efficiency could render them basically worthless), but they literally have the worthless aspect built into them directly. You will not be able to profitably run an LLM on a 5 year old GPU, as your competitors will be able to run inference at much higher efficiencies than you with modern chips and will undercut you on price.
It is just different because it is an almost guaranteed point in the future that they become worthless (unlike rail or real estate).
> GPU's are interesting in that they are like tulips (a breakthrough in efficiency could render them basically worthless), but they literally have the worthless aspect built into them directly.
Don't tulips have the "worthless aspect" AKA end-of-life built in as well? They perish, after all!
> You will not be able to profitably run an LLM on a 5 year old GPU, as your competitors will be able to run inference at much higher efficiencies than you with modern chips and will undercut you on price.
If you're in the business of selling tokens, certainly. If you're in the business of something else, and use LLMs to speed a process up, do you care that the 1-day process now takes 2 hours on 5yo hardware and 1 hour on SOTA hardware?
The only businesses in trouble here are those in the business of selling tokens. Those businesses selling potatoes who use an LLM to streamline/shorten some business process aren't going to care.
This comment reminds me of this paper which makes a similar argument. It claims that chips are more than just a depreciating asset you actually have to keep buying more for the economics to enable the scaling. And inherently there's going to be ups and downs. The AI labs seem to not be preparing for the depreciation nor the scaling requirements and that puts their economics into question.
https://gauthierroussilhe.com/en/articles/how-to-use-computi...
5 years is an eternity in this biz though