> AI will end up being just slightly more expensive than the raw electricity required to produce it.
AFAICT, this hasn't happened for any "cloud" compute company that has been in business for -say- at least five years. Unless your prediction is that roughly no one will use "cloud" "AI", I'm rating this prediction as "totally bogus". And if that is your prediction, then what you're saying is
Assuming that you're not doing anything that needs exotic system administrators, nearly all of the cost of computing when you own your own hardware is the power for that hardware.
which... unless you're buying used hardware, I don't think that's true, either. Hetzner quotes a max of 167.23 EUR/month for a full rack and what I think is 2400W of power, and lists the rate for Finnish power as 0.22 EUR per kWh. [0] If we assume that Hetzner's spending cap doesn't apply and we are forced to pay the full quoted rate for power in Finland, I think that works out to a bit under 4.6K EUR per year:
240V * 10A = 2.4kW
2.4kW * 24H * 30 * 12 = 20,736 kWh
20,736kWh * 0.22 EUR/kWh = 4562 EUR/year
The last time I got quotes for most of a rack of server hardware was like a decade ago, and that pretty-bog-standard server hardware ran around 200K USD. If we assume a ten-year expected life [1], that hardware cost about 17K EUR per year. Assuming I haven't fucked up my math, [2][3] the cost of the hardware here is more than three times the cost of power.
Note that I'm using roughly decade-old hardware prices and allegedly-current electricity prices. If we used hardware prices from two years ago -or worse, from today- the hardware would cost far more per year.
[0] <https://www.hetzner.com/colocation/>
[1] ...which I think is between two and three times longer than is standard to expect in the business world...
[2] ...it's very likely that I have! Corrections are definitely welcome.
[3] ...it's also possible that USians get gouged for server hardware and that it's like a quarter of the cost if you buy it in Finland. I expect that there's definitely a price difference, but I doubt that it's very large.
>AFAICT, this hasn't happened for any "cloud" compute company that has been in business for -say- at least five years.
Most cloud software has had moats. AI does not. And AI is eating software, which means those historical margins are going away for a lot of cloud software,too. That's what the SAAS-pocalypse is about.
>Assuming that you're not doing anything that needs exotic system administrators
Remember, these are also going to be AI.
>Assuming I haven't fucked up my math, [2][3] the cost of the hardware here is more than three times the cost of power.
This is a fair point, but it just means the COGS is higher, not that the margin is any better. I stand by the prediction that AI margins will be very slim.
> I stand by the prediction that AI margins will be very slim.
This I don't disagree with... but you and I probably believe this for very different reasons. However, the rest of what you've said -in this comment and the first one I replied to- is just bonkers word salad.
> is just bonkers word salad.
I think you're placing much too much emphasis on the "electricity" comment, which I admit was an overstatement. But the description of the economics driving data center deals is quite parseable for someone familiar with the financial structure of the AI industry.
As for the rest, if you point out the parts that you don't understand, I'd be happy to elaborate.
> I think you're placing much too much emphasis on the "electricity" comment...
No. I addressed your electricity assertion here. [0] I'm calling nearly all of your commentary word salad because it reads like the "conversation" in the picture behind the "...cognitotoxic" cut on this Mastodon post [1], just with the "spiritual woo" vocabulary module pulled out and the "finance bro" and "LLM Tech Exuberance" vocabulary modules loaded in.
[0] <https://news.ycombinator.com/item?id=50203639>
[1] <https://neuromatch.social/@jonny/117492479270310799>
Ha, now that's a word salad.^^^^^ Well done, I appreciate a well-crafted ironic troll as much as the next man.
Even if that scenario did happen (I agree I don't think most of the major providers would survive on a commodity utility model). Most of the advances in AI are driven by huge amounts of funding for research. If everyone goes to open source models/utility based pricing etc then it'll be cheaper but the current rate of improvements are being overwhelmingly driven by the frontier labs (for better or worse) and they're being driven by vast amounts of VC funding predicated on making vast amounts of money.
I'm very dubious we can have our cake and eat it of constant magnitude order improvements and also commodified utility pricing.
What? You're saying that Hetzner is losing money?
168 * 12 < 4562
That is correct. It's a little under half that. It's why I said
> You're saying that Hetzner is losing money?
If I were saying that, I would have said it. Unless "XXX max/mo." doesn't mean what it means in plain English, I _very_ strongly expect that it's very rare for colo customers to suck down all available power 24/7 and that Hetzner has a Stern Talk with customers who do but didn't disclose their intent to do so up front. I mean, you tell me... if you've ever lived in an apartment where your power and water was included in the rent and was metered with a whole-building meter (rather than per-unit meters), did you consume every amp and gallon that could be delivered to you, or did you use just a bit more than what was useful to you in the course of everyday life? [0]
I worked the math the way I did (even doubling or tripling the expected lifetime of the server hardware) to make the numbers as favorable for the "electricity is almost 100% of the cost of computing" side of the argument as they could reasonably be.
Are you saying that I fucked up my math?
[0] And if you're never lived in such an arrangement, ponder instead how you would behave if you did find yourself in one.
Fair. Your argument is that the price of the hardware dominates the price of electricity in cloud computing
> Your argument is that the price of the hardware dominates the price of electricity in cloud computing.
No. The numbers indicate that the price of all "full-sized" computing is mostly the cost of hardware, [0] regardless of whether that's "cloud" computing, "on prem" computing, or -er- "colocated" computing.
And by "full-sized" I mean "computing with regular PC or server-type hardware". I'm sure someone could build some absurdly cheap exotic stuff that invalidates all this math, but I don't expect anything that will be found in a datacenter, server room, or on someone's desk at work and whose primary job is doing compute that customers are paying for to be that stuff.
[0] Unless that hardware is acquired at a _deep_ discount... which usually isn't possible if you're buying new hardware. And if you're buying used hardware at a deep discount, that usually means that someone paid a normal price for that hardware at some point.