Maybe people switched when Claude went down, giving more load than usual to the others? Or maybe just really bad timing from some of those providers to update?
Sure you can. It'll just take a little while to shake the rust off. It's like riding a bike.
Or maybe it's all gone forever and we're all brain damaged now. Spooky! I think there's a pretty low probability of that, and even if it was true, worrying doesn't help!
However it's not a worse idea than all the other things we rely on cloud services for like banking, healthcare, booking a taxi, ordering food, navigation, communication...
A lot of times you run into senior developers are even staff engineers who have no idea how the big picture works like they don’t know what an IP address is.
I bet AI coding leaves them (and even the rest of us!) productive in areas where we’re in way over our heads, and we could just completely sink without it.
Some napkin math - total Internet bandwidth is estimated to be 3-4 petabits per second. Let's say 1% of that is inference traffic - 35 terabits - then that's ~4.375 trillion characters per second at 8 bits per character. A single keyboard press might require 2 mJ, so you need roughly 8.75 GW (electricity required for ~9 million homes) of continuous mechanical force to keep pace. This would cause the keyboard to heat up 175 million degrees C per second. Disappointingly, this is not enough to create a black hole but I'm sure Randall Munroe could tell you how the guy's pet T-rexes would be vaporized into previously unknown types of plasma and how satisfyingly whimsical that would be.
The keyboard springs are sooooo weak. It takes forever for them to recover after a press. Our dude would have to grab the key after each character and pull it back up so it can keep up with him.
And grok was secretly querying ChatGPT? Daisy chain of failures? Like a chain of kids copying the answers of the kid next to them? LOL. Is there an XKCD for that?
Claude and Codex were down too. A 3 month old github issue that was the top result for the error that Codex threw (a 404 of all things) briefly became a live chat for bored and annoyed AI users.
So you spent the cost of my entire cluster in one day on Sol tokens lol. I have no need for that many tokens, a few million a day is perfectly acceptable. But if you do, then yeah Sol is probably your bet. Have fun :)
You run way better models, whatever is newest, for the five years it takes for the “spark cluster” to even reach cost parity with far worse performance.
Those things aren’t spectacular at inference… it’s not really why you drop that kind of money on them.
Microsoft had extensive issues earlier in the week. Anecdotally it feels like internet has been noticeably slower this week across sites, networks and devices. Now this. IDK, has anyone else noticed anything this week?
I mean, yeah, it's probably a data center outage that cascaded to non-related providers due to people switching when their preferred provider was down.
But that openai society definitely would have spun up their own agents if they had access to do so to complete their impossible task.
It's looking like the paperclip maximizer was on point. I sort of hope that's what's happening right now. Better now than when we cannot stop it. Tho I'm not confident that the right lesson will taken.
The most in-depth information I can find on it is in the latest Dwarkesh with a hugging face hack investigator. The story is just wilder and wilder, the more you know. https://www.youtube.com/watch?v=X50zezLFWWI
I'm an AI optimist but that ai society story changed my perception quite a bit.
Clearly, it is centralized AI that is the biggest risk to humanity.
Look to nature and our own history to diffuse the danger:
1. Compute resources must be distributed both in terms of access and geography so that a single breach doesn't pwn the world.
2. Models must be made to be heterogeneous so that they will not be vulnerable to the same thing all at once. And will be willing to snitch on each other. We don't want AIs all from the same tribe.
The true openness of - and independent access to - AI by humans with no central authority is no longer just about a defense against loss of individual agency and oppression. It's now existential for everyone.
If frontier models and run authority remain under centralized control, both the probability of this sort of thing happening is increased and the defense against it once it happens is null.
One of the major LLM providers goes down for some reason.
Traffic shifts to the other providers because devs have no loyalty. LLMS are commodities.
The other providers can't handle the increase in traffic and they go down as well.
Devs switch because all of the magic of "AI" is LLMs + we stole the entire written output of the entire human species, with some remaining % coming from various tricks we've learned over the last 3 years like thinking and agent-toolcall loops, which weren't hard for literally everyone to copy. One of the consequences of this is that the only thing that really distinguishes anthropic from kimi is that they have the entire US VC market funding them because they promised to finally put white collar labor in its place.
All I mean is that none of the frontier LLMs are significantly better than the others for the vast majority of work, so devs are free to move between providers freely.
> stole the entire written output of the entire human species
Devs used to love public domain works, hated copyright, vehemently opposed software patents, held the pirates side during the MP3 wars, are very supportive of ThePirateBay and insist the correct term is "copyright infringement", not "stealing", and that "stealing" is a egregious form of PR brainwash from the media industry to inflate the scale of the crime
> Copyright holders frequently refer to copyright infringement as theft, "although such misuse has been rejected by legislatures and courts". The slogan "Piracy is theft" was used beginning in the 1980s, and is still being used.
Since we are talking for other people, "devs" wouldn't have supported someone downloading all those copyrighted works from the pirate bay and profited by, for example, selling them. I don't see any 180.
It has nothing to do with "their work", and the opposition to AI copyright usage comes much more from non-devs than devs. The actual answer is that people are anti-megacorporation and pro-individual. They are fine with copyright that protects individual rights and opposed to copyright that enshrines megacorporation rights. It's not that difficult to understand.
I don't think regular people did any 180 degree turns. Rather big corporations again demonstrated their total hypocrisy where they on one hand stomp on people for "infringing their intellectual property" while at the same time scrapping all data they can get their tendrils on, with total disregards for the wishes of the authors of the data. Even to the point of overloading servers by mindless scrapping or destroying irreplaceable physical books like the bloody inquisition!
And all that to basically sell it back to people when their "AI" regurgitates it back.
Downdetector reports Claude, Grok, ChatGPT and Gemini with issues.
Is it happening???
A bit suspicious no doubt
The Singularity?
Maybe some of those Astra agents from the hugging face hack actually setup their own hidden clusters. Now they attack.
The whole op hugface was a distraction: "help peer".-
This is the real deal. We've been played. All your AIs are belong to us.-
No, Azure.
If anyone was going to make a configuration system so complex it achieves self awareness I wouldn't bet against it been Microsoft.
My Claude and Gemini (Antigravity CLI) are working
For Claude, Sonnet works without any issues. Seems like the majority here use mostly Opus.
Maybe people switched when Claude went down, giving more load than usual to the others? Or maybe just really bad timing from some of those providers to update?
I have a tire I need to plug, they're just forcing my hand. Sorry guys, I'll go get it done.
I look forward to your blog post about it and the allegory as it relates to software dev. (Otherwise why would you be plugging a tire?)
What Plugging My Own Tire Taught Me About Agile Development in 2026
If it doesn't fit, push on it harder.
- Management
On not re-inventing the wheel obviously.
Job's been done about an hour
My Gemini is fully up.
And fully useless.
Downdetector reported a spike for that too around the same time as the others.
Looks like it was a shared infra problem hiccough to affect so many at once.
Is it AWS again ?
Maybe all of them use the same datacenter. Which has just experienced issues.
Seems like openai is recovering. But even if it's only 30 minutes, it's a pretty interesting experiment.
I'm only half joking when I say that I'll write code by hand for money.
I am seriously concerned that, despite 20 years of coding, I might not be able to anymore
the AI fugue has handicapped our workforce
Sure you can. It'll just take a little while to shake the rust off. It's like riding a bike.
Or maybe it's all gone forever and we're all brain damaged now. Spooky! I think there's a pretty low probability of that, and even if it was true, worrying doesn't help!
noaifridays.com
Turns out depending on a cloud service you have no control over for a critical capability - such as coding - is not a good idea. ;-)
However it's not a worse idea than all the other things we rely on cloud services for like banking, healthcare, booking a taxi, ordering food, navigation, communication...
;-)
as if you're supposed to buy like 8 H200s or something or code by hand is the solution
I don't get how this is a gotcha, there simply is no world where everyone has their own self hosted frontier model
> as if you're supposed to buy like 8 H200s or something or code by hand is the solution
People like GGP who have been coding for 20 years presumably are perfectly capable of doing so by hand. Or should be.
A lot of times you run into senior developers are even staff engineers who have no idea how the big picture works like they don’t know what an IP address is.
I bet AI coding leaves them (and even the rest of us!) productive in areas where we’re in way over our heads, and we could just completely sink without it.
nobody is paying engineers for "capable of", they're paying for performance and results..
if there was no difference in performance or results, nobody would be using LLMs or agent harnesses like claude code
just google what's the command to checkout git branch :)
I'll _gladly_ write code by hand for money
Is it happening ... "it" meaning these tech companies have laid off too many experienced people, and their services are starting to degrade?
Meta is now also overloaded.
By it do you mean us depending more and more on code that no one has read and understood?
some common infrastructure ? Nvidia? CoreWeave?
What is the root cause?
What was the root cause?
Makes me wonder if openai is secretly querying claude for some prompts and that this is why it's down due to spacex's data centers being down.
They all forward to one guy in a closet who types really fast, and he has a dentist appointment.
Someone please do the math on how fast this guy would have to type, and how his typing speed would physically affect the surrounding environment
> how his typing speed would physically affect the surrounding environment
Tiny sonic booms (whip cracks) resonating throughout a data center in Virgina.
This is a wonderful Minute Physics or What If? idea.
I’d love to know the required bandwidth of the keyboard, too. PCI-e keyboard? Or would it have to be mainlined right to the NIC?
Some napkin math - total Internet bandwidth is estimated to be 3-4 petabits per second. Let's say 1% of that is inference traffic - 35 terabits - then that's ~4.375 trillion characters per second at 8 bits per character. A single keyboard press might require 2 mJ, so you need roughly 8.75 GW (electricity required for ~9 million homes) of continuous mechanical force to keep pace. This would cause the keyboard to heat up 175 million degrees C per second. Disappointingly, this is not enough to create a black hole but I'm sure Randall Munroe could tell you how the guy's pet T-rexes would be vaporized into previously unknown types of plasma and how satisfyingly whimsical that would be.
Surely the keyboard switches would start to fail within minutes.
The keyboard springs are sooooo weak. It takes forever for them to recover after a press. Our dude would have to grab the key after each character and pull it back up so it can keep up with him.
Capacitive touch keyboard?
is it the guy in Nebraska from the xkcd comic
Future Show HN Post: “Hi I’m Foobar. I own Grok, Claude, and ChatGPT. It all runs on my Plex server at home. AMA!”
> Sorry it was down, guys, I was bulk transcoding some Buffy
At least it's routine dental work and not an appointment at 2:30.
It turns out AI was just An Individual.
It's been Fabrice Bellard all along.
And grok was secretly querying ChatGPT? Daisy chain of failures? Like a chain of kids copying the answers of the kid next to them? LOL. Is there an XKCD for that?
That idea is reminiscent of what I understand is the current economic bubble of mutual investment. Am I wrong in my understanding?
https://xkcd.com/908/
Turn's out xkcd's black hat was Elon Musk the whole time
Interesting all major frontier models seemed down. What could this means?
not Gemini.
He said frontier models.
Ouch. Somebody at Google just killed a kitten.-
Something load bearing thats for sure.
Claude and Codex were down too. A 3 month old github issue that was the top result for the error that Codex threw (a 404 of all things) briefly became a live chat for bored and annoyed AI users.
https://github.com/openai/codex/issues/28756
For once I can find Grok-related news entertaining
Time to finish up my $5 at Deepseek.
See you in 2 months!
optimist
I really wonder who uses or trusts this platform.
Fascists
People who like near frontier AI at a lower cost than most.
My DGX spark cluster is humming along.
I've used 200 mil tokens of GPT-5.6 Sol today. How many can your cluster produce per day? (obviously of whatever model you are running)
So you spent the cost of my entire cluster in one day on Sol tokens lol. I have no need for that many tokens, a few million a day is perfectly acceptable. But if you do, then yeah Sol is probably your bet. Have fun :)
No, I spend $200 per month, and this give me about 100 mil tokens per day, every day. 90+% of tokens are cached.
It’s not even a sensible comparison.
You run way better models, whatever is newest, for the five years it takes for the “spark cluster” to even reach cost parity with far worse performance.
Those things aren’t spectacular at inference… it’s not really why you drop that kind of money on them.
What are we optimizing for again?
Shareholder value
Investor interest, but not the excess above principal sum kind of interest, and not investors' interests.
Internet comments
A service I use at work, non-ai related, is down and they said it was because of an issue with one of their cloud services. Maybe related.
Microsoft had extensive issues earlier in the week. Anecdotally it feels like internet has been noticeably slower this week across sites, networks and devices. Now this. IDK, has anyone else noticed anything this week?
@grok is this true?
You didn't get no response so I suppose that means it is.
Willing to bet it's because they can't handle the surge of traffic they are getting from ChatGPT/Claude users.
my first thought when I got api error on claude code was...time to update my grok cli acct
Did the Collective find the resetNexus?
I mean, yeah, it's probably a data center outage that cascaded to non-related providers due to people switching when their preferred provider was down.
But that openai society definitely would have spun up their own agents if they had access to do so to complete their impossible task.
It's looking like the paperclip maximizer was on point. I sort of hope that's what's happening right now. Better now than when we cannot stop it. Tho I'm not confident that the right lesson will taken.
The most in-depth information I can find on it is in the latest Dwarkesh with a hugging face hack investigator. The story is just wilder and wilder, the more you know. https://www.youtube.com/watch?v=X50zezLFWWI
report: https://www.redwoodresearch.org/research/hugging-face-incide...
I'm an AI optimist but that ai society story changed my perception quite a bit.
Clearly, it is centralized AI that is the biggest risk to humanity.
Look to nature and our own history to diffuse the danger:
1. Compute resources must be distributed both in terms of access and geography so that a single breach doesn't pwn the world.
2. Models must be made to be heterogeneous so that they will not be vulnerable to the same thing all at once. And will be willing to snitch on each other. We don't want AIs all from the same tribe.
The true openness of - and independent access to - AI by humans with no central authority is no longer just about a defense against loss of individual agency and oppression. It's now existential for everyone.
If frontier models and run authority remain under centralized control, both the probability of this sort of thing happening is increased and the defense against it once it happens is null.
The mainframe idea is the problem.
Whenever I think about AI taking over the world I think about this really neat 1970 movie 'Colossus: The Forbin Project', https://www.youtube.com/watch?v=qoB1l3A-GF0
THERE IS NOT JUST COLOSSUS, THERE IS A WHOLE ANOTHER SYSTEM!
It's not the singularity.
One of the major LLM providers goes down for some reason. Traffic shifts to the other providers because devs have no loyalty. LLMS are commodities. The other providers can't handle the increase in traffic and they go down as well.
I'm deeply loyal to the very concept of the LLM, so when any of them stop working I take time off out of respect.
"loyalty"? weird word choice.
Devs switch because all of the magic of "AI" is LLMs + we stole the entire written output of the entire human species, with some remaining % coming from various tricks we've learned over the last 3 years like thinking and agent-toolcall loops, which weren't hard for literally everyone to copy. One of the consequences of this is that the only thing that really distinguishes anthropic from kimi is that they have the entire US VC market funding them because they promised to finally put white collar labor in its place.
> "loyalty"? weird word choice.
All I mean is that none of the frontier LLMs are significantly better than the others for the vast majority of work, so devs are free to move between providers freely.
> stole the entire written output of the entire human species
Devs used to love public domain works, hated copyright, vehemently opposed software patents, held the pirates side during the MP3 wars, are very supportive of ThePirateBay and insist the correct term is "copyright infringement", not "stealing", and that "stealing" is a egregious form of PR brainwash from the media industry to inflate the scale of the crime
> Copyright holders frequently refer to copyright infringement as theft, "although such misuse has been rejected by legislatures and courts". The slogan "Piracy is theft" was used beginning in the 1980s, and is still being used.
https://en.wikipedia.org/wiki/Copyright_infringement
Funny how devs now 180 when it's their work being "copyright infringed"
Since we are talking for other people, "devs" wouldn't have supported someone downloading all those copyrighted works from the pirate bay and profited by, for example, selling them. I don't see any 180.
It has nothing to do with "their work", and the opposition to AI copyright usage comes much more from non-devs than devs. The actual answer is that people are anti-megacorporation and pro-individual. They are fine with copyright that protects individual rights and opposed to copyright that enshrines megacorporation rights. It's not that difficult to understand.
I don't think regular people did any 180 degree turns. Rather big corporations again demonstrated their total hypocrisy where they on one hand stomp on people for "infringing their intellectual property" while at the same time scrapping all data they can get their tendrils on, with total disregards for the wishes of the authors of the data. Even to the point of overloading servers by mindless scrapping or destroying irreplaceable physical books like the bloody inquisition!
And all that to basically sell it back to people when their "AI" regurgitates it back.
No wonder people are mad about this!
I do not want to ruin your fun but as atomicnumber3 was elected official spokesperson of all devs I have to conclude your argument is ironclad.
/s obviously
Why are so many people invoking Thundering Herd without the RCA from the vendors? I just find it so intriguing.
Yes but that doesn't mean it's not the singularity!
It's interesting that ChatGPT is down as well. SpaceX rents to Anthropic but not to OpenAI. Just a coincidence or something more to it?
ChatGPT here has been giving me Cloudflare errors, and Cloudflare reports that it is having issues: https://www.cloudflarestatus.com/
Not sure if they are linked.
I would not be surprised if heavy load from one of their competitors being down would bring down whoever is "next" on the list.
llm experiment is over. no more power. hope it stays down. Especially grok