I am mainly mentioning this with regards to Azure and other providers egress prices.
And in Europe, onprem stuff is expensive if you are peering to other countries.
The last time I had to care professionally about bandwidth pricing for CDN price optimization in the US, wholesale bandwidth pricing was following a pattern similar to Moore’s law, with either bandwidth doubling, or price halving every 18-21 months. This was partly why you could get what looked like good deals from CDN providers for multi year contracts. They knew their prices were just going to fall. Part of what drives this is that we keep finding ways to utilize fiber, so there’s a technical aspect, but a lot of it also comes down to adding more physical connections. There’s even network consolidation happening where 2 companies will do enough data sharing that they will get peering agreements and just add a cat6 patch between servers hosted in the same datacenter and short circuit the network.
It’s been almost a decade so it’s possible things have slowed considerably, or demand has outstripped supply, but given how much data steam seems to be willing to throw at me, I know pricing is likely no where near what it was last I looked (it’s the only metered thing I regularly see and it’s downloading 10’s of GB daily for a couple games in my collection).
Using egress pricing is also the wrong metric. You’d be better off looking at data costs between regions/datacenters to get a better idea about wholesale costs, since high egress costs is likely a form of vender lockin, while higher looking at cross region avoids any “free” data costs through patch cables skewing the numbers.
Not sure about bandwidth between countries, there’s different economics there. I’d expect some self similarity there, but laying trunks might be so costly that short of finding ways to utilize fiber better is the only real way to increase supply.
Azure and the other mega clouds seem to enjoy massive profit margins on bandwidth… why would they willingly drop those prices when they can get away with high prices?
If bandwidth costs are important, there are plenty of options that will let you cut the cost by 10x (or more). Either with a caching layer like an external CDN (if that works for your application), or by moving to any of the mid-tier clouds (if bandwidth costs are an important factor, and caching won’t work for your application).
AWS, GCP, and Azure are the modern embodiment of the phrase “nobody ever got fired for buying IBM.”
Most companies don’t benefit from those big 3 mega clouds nearly as much as they think they do.
So, sure, send a note to your Azure rep complaining about the cost of bandwidth… nothing will change, of course, because companies aren’t willing to switch away from the mega clouds.
> and other providers
Other providers, like Hetzner, OVH, Scaleway, DigitalOcean, Vultr, etc., do not charge anywhere near the same for bandwidth as Azure. I think they are all about 8x to 10x cheaper.
A CDN will increase your bandwidth costs not lower it.
Eg Fastly prices:
US/Europe $0.10/GB
India $0.28/GB
Not all bandwidth is equal. eg Hetzner will pay for fast traffic into Europe but don't pay the premium that others like AWS do to ensure it gets into Asia uncongested.
BunnyCDN charges significantly less for data that they serve, for example.
I didn’t say all CDNs are cheaper. Some CDNs see an opportunity to charge a premium, and they do!
Fastly sees themselves as far more than just a CDN. They call themselves an “edge cloud platform”, not a CDN.
> Not all bandwidth is equal. eg Hetzner will pay for fast traffic into Europe but don't pay the premium that others like AWS do to ensure it gets into Asia uncongested.
Sure… there are sometimes tradeoffs, but for bandwidth-intensive apps, you’re sometimes (often?) better off deploying regional instances that are closer to your customers, rather than paying a huge premium to have better connectivity at a distance. Or, for CDN-compatible content, you’re probably better off using an affordable CDN that will bring your content closer to your users.
If you absolutely need to use AWS’s backbone for customers in certain geographic regions, there’s nothing stopping you from proxying those users through AWS to your application hosted elsewhere, by choosing the AWS region closest to your application and putting a proxy there. You’ll be paying AWS bandwidth plus your other provider’s bandwidth, but you’ll still be saving tons of money to route the traffic that way if those geographic regions only represent a small percentage of your users… and if they represent a large percentage, then you can host something more directly in their region to make the experience even better.
For many types of applications, having higher latency / lower bandwidth connectivity isn’t even a problem if the data transfer is cheaper and saves money… the application just needs to do better caching on the client side, which is a beneficial thing to do even for clients that are well-connected to the server.
It depends, and I am not convinced there is a one-size-fits-all solution, even if you were to pay through the nose for one of the hyperscalers.
I have plenty of professional experience with AWS and GCP, but I also have professional experience with different degrees of bare metal deployment, and experience with mid-tier clouds. If costs don’t matter, then sure, do whatever.
Transit is cheap (and gets cheaper every year), cloud markups and profit margins are expensive. Like, you can still rack a server and pay peanuts for the networking, but that isn't covered in a Medium post, so nobody knows how to do it anymore.
I love how everyone is arguing about networking costs inside the tiny prison cell is "the cloud". Because obviously the only way to push bits over the wire is through an AWS Internet Gateway, which was the very first packet-switched routing ever.
How much do you think bandwidth costs? I’m trying to understand what 10x cheaper would look like to you, as an actual $/TB number.
I think a lot of people have misconceptions about how much bandwidth really costs.
I am mainly mentioning this with regards to Azure and other providers egress prices. And in Europe, onprem stuff is expensive if you are peering to other countries.
The last time I had to care professionally about bandwidth pricing for CDN price optimization in the US, wholesale bandwidth pricing was following a pattern similar to Moore’s law, with either bandwidth doubling, or price halving every 18-21 months. This was partly why you could get what looked like good deals from CDN providers for multi year contracts. They knew their prices were just going to fall. Part of what drives this is that we keep finding ways to utilize fiber, so there’s a technical aspect, but a lot of it also comes down to adding more physical connections. There’s even network consolidation happening where 2 companies will do enough data sharing that they will get peering agreements and just add a cat6 patch between servers hosted in the same datacenter and short circuit the network.
It’s been almost a decade so it’s possible things have slowed considerably, or demand has outstripped supply, but given how much data steam seems to be willing to throw at me, I know pricing is likely no where near what it was last I looked (it’s the only metered thing I regularly see and it’s downloading 10’s of GB daily for a couple games in my collection).
Using egress pricing is also the wrong metric. You’d be better off looking at data costs between regions/datacenters to get a better idea about wholesale costs, since high egress costs is likely a form of vender lockin, while higher looking at cross region avoids any “free” data costs through patch cables skewing the numbers.
Not sure about bandwidth between countries, there’s different economics there. I’d expect some self similarity there, but laying trunks might be so costly that short of finding ways to utilize fiber better is the only real way to increase supply.
Azure and the other mega clouds seem to enjoy massive profit margins on bandwidth… why would they willingly drop those prices when they can get away with high prices?
If bandwidth costs are important, there are plenty of options that will let you cut the cost by 10x (or more). Either with a caching layer like an external CDN (if that works for your application), or by moving to any of the mid-tier clouds (if bandwidth costs are an important factor, and caching won’t work for your application).
AWS, GCP, and Azure are the modern embodiment of the phrase “nobody ever got fired for buying IBM.”
Most companies don’t benefit from those big 3 mega clouds nearly as much as they think they do.
So, sure, send a note to your Azure rep complaining about the cost of bandwidth… nothing will change, of course, because companies aren’t willing to switch away from the mega clouds.
> and other providers
Other providers, like Hetzner, OVH, Scaleway, DigitalOcean, Vultr, etc., do not charge anywhere near the same for bandwidth as Azure. I think they are all about 8x to 10x cheaper.
A CDN will increase your bandwidth costs not lower it.
Eg Fastly prices: US/Europe $0.10/GB India $0.28/GB
Not all bandwidth is equal. eg Hetzner will pay for fast traffic into Europe but don't pay the premium that others like AWS do to ensure it gets into Asia uncongested.
BunnyCDN charges significantly less for data that they serve, for example.
I didn’t say all CDNs are cheaper. Some CDNs see an opportunity to charge a premium, and they do!
Fastly sees themselves as far more than just a CDN. They call themselves an “edge cloud platform”, not a CDN.
> Not all bandwidth is equal. eg Hetzner will pay for fast traffic into Europe but don't pay the premium that others like AWS do to ensure it gets into Asia uncongested.
Sure… there are sometimes tradeoffs, but for bandwidth-intensive apps, you’re sometimes (often?) better off deploying regional instances that are closer to your customers, rather than paying a huge premium to have better connectivity at a distance. Or, for CDN-compatible content, you’re probably better off using an affordable CDN that will bring your content closer to your users.
If you absolutely need to use AWS’s backbone for customers in certain geographic regions, there’s nothing stopping you from proxying those users through AWS to your application hosted elsewhere, by choosing the AWS region closest to your application and putting a proxy there. You’ll be paying AWS bandwidth plus your other provider’s bandwidth, but you’ll still be saving tons of money to route the traffic that way if those geographic regions only represent a small percentage of your users… and if they represent a large percentage, then you can host something more directly in their region to make the experience even better.
For many types of applications, having higher latency / lower bandwidth connectivity isn’t even a problem if the data transfer is cheaper and saves money… the application just needs to do better caching on the client side, which is a beneficial thing to do even for clients that are well-connected to the server.
It depends, and I am not convinced there is a one-size-fits-all solution, even if you were to pay through the nose for one of the hyperscalers.
I have plenty of professional experience with AWS and GCP, but I also have professional experience with different degrees of bare metal deployment, and experience with mid-tier clouds. If costs don’t matter, then sure, do whatever.
Or what they’re actually buying when they’re looking at the bandwidth line item on their invoices.
They charge you for value, not for cost plus. Bandwidth is cheap if all you're getting is bandwidth.
Transit is cheap (and gets cheaper every year), cloud markups and profit margins are expensive. Like, you can still rack a server and pay peanuts for the networking, but that isn't covered in a Medium post, so nobody knows how to do it anymore.
bandwidth is cheap as hell
egress in the cloud is deliberately expensive as an anti-competitive measure to lock you in and stop you using competitors services
"They hated him because he spoke the truth."
I love how everyone is arguing about networking costs inside the tiny prison cell is "the cloud". Because obviously the only way to push bits over the wire is through an AWS Internet Gateway, which was the very first packet-switched routing ever.