This experience, which you've described well in just one sentence, is actually a consequence of how the site is structured. You might say "well, structure it differently then, so it doesn't suck". But giving up that structure would be bad, because although it is far from pleasant, it contains something that's both valuable and increasingly rare. What we need to do instead is find our way through the difficulty without cracking up. That's the question facing the community.
HN is divided much as society at large is divided, just like any large-enough population sample. It's also a highly international site. Cultural norms vary more than people assume they do, especially when you factor in the international aspects. Since the debate and struggle right now is around just what "current cultural norms" ought to be, it isn't surprising that commenters disagree.
The most important thing to realize about HN is that it's a non-siloed site. Everyone is in one big room here. That means you're far more likely to run into people who hold opposing views on these topics, and they may be the very same people who you agree with about other topics. This is very different from internet environments where users self-select into tribes (silos) and where the tribes are clearly identifiable. There, you know what awfulness to expect from the other side, you can dress up in the battle gear of the internet before confronting them, and you know that your own team will have your back.
On HN, by contrast, you don't know what awfulness to expect, because the tribes are not identifiable and the community actually functions relatively well some of the time (don't get me wrong, it's the internet and there are a ton of problems—but, relatively speaking). This creates a feeling of normalcy and so people expect this place to be normal ("current cultural norms", as you say). Some of the time it does indeed feel that way, and then—blam, you run into something awful and hideous, something anything but normal, which you did not expect to encounter here and which hit you when your defenses were down.
This comes across as a nasty shock. After a few of these shocks, the mind inevitably adjusts its view of the community from "normal place" to "painful nasty place", with the sense "HN is not what I thought it was". I'm going to guess that this is the reason why you use the phrase "astounds me".
In other words, we're in a paradoxical situation where the HN community is objectively less divided and nasty than other places on the internet (which "solve" or, more precisely, defer this problem by segregating into silos), but it's still divided and nasty enough to create unexpectedly painful experiences, which makes it feel more divided and nasty.
This has probably always been the biggest issue for HN's long-term survival, but it has taken years to begin to realize it, and of course current social pressures are intensifying and highlighting it. It's not clear yet what we can really do, let alone if it can survive in the long run, but from the beginning this place has been an experiment in staving off self-destruction for as long as possible (https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...).
I wrote another thing about the shock experience here if anyone is interested: https://news.ycombinator.com/item?id=23308098
It's not really all that hard to know what awfulness to expect.
edit: That probably sounds more dickish than I meant it. I don't know a better place than HN for identifying clusters and following trends in the thought of "temporarily embarrassed millionaires".
It's not only that, which is why I still come here. But that's the strongest theme by far.
> strongest theme by far
Such perceptions are notoriously unreliable. People notice different things and weight them differently, based mostly on what they dislike: https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que.... Since everyone dislikes different things, everyone has a different weighted average, each reflecting their own preferences. In other words they are in the eye of the beholder, which explains why people come up with such wildly contradictory generalizations about HN.
From my perspective, that's the strongest theme by far. Does my argument apply to my own perception? For sure. It has to.
The “tribes” are easily identifiable on HN, it seems to me. For all of the most prolific HN posters, it is quite easy to predict which side of nearly any issue they’ll support.
Good point, but it takes a lot of familiarity before those lines become clear. At that point, the shock experience gets weaker, there is more a sense of "I expect that from this asshole", and the discussions become more like the ant wars on the rest of the internet.
Even then, the community is big enough that you're going to regularly run into comments from users you either don't recognize, or didn't realize were of the opposing tribe (or opposing-tribe-adjacent, let's say). So the shock experience never goes away. But I think you're right that it's most intense for less-familiar users, and that explains why there are so many people who use HN for a little while, inevitably encounter the shock experience, feel hurt, and leave. That's a common occurrence and which pains me personally.
The deeper point is that the site is designed to keep everybody together. It's not partitioned by friends or follow lists. There are no subreddits or social graphs. Design differences at that level are profound and affect everything.