I'm sorry, but no, and as a mutt user myself: email sucks.
Spam. The protocol is insecure. Consolidation of major webmail providers is a huge problem on multiple fronts (censorship, third-party acceptance (of both delivery and non-GmailMicrosoftYahoo email accounts --- my Protonmail service can't talk to GetPocket's support email on Gmail for months, f'rex), spam, account hacking, authentication, repudiability (yes, that coin has two sides), workflow integrations, proprietary workflows, spam, account lockouts, lack of common email formatting and practice standards, HTML email, spam, pervasive surveillance, attachments, spam, mobile access, spam, spam, infinite diversity in proprietary messaging systems and platforms each fractally worse than email in a myrad ways, spam, spam, spam, inbox overflow, spam, spam, spam, spam, email bankruptcy.
I've had an email account nearly continuously since the mid-1980s. I've all but abandoned it in the past decade.
There is no viable replacement on the horizon, though postal mail is looking increasingly attractive.
I am happy to deal with the spam problem (through appropriate use of spamassassin content scoring, and RBLs and spamassassin DKIM/SPF scoring) on my own smtpd in exchange for the great advantage of having a truly decentralized communication method that isn't controlled by any one company.
My email works as long as I keep my domain name renewed and my personal postfix+dovecot server online and running properly.
Nearly every other possible asynchronous, text based two way communications system that people say is a "possible replacement for email" requires centralizing communications through something run by some specific company.
Email's great advantage is that it's a truly level playing field. Yes that also allows spammers to send outgoing shit smtp traffic, but I can deal with that.
If you are greatly troubled by spam and it's an actual ongoing problem in your inboxes I would say that your inbound smtp content filtering and spam scoring is not set up properly.
> Email's great advantage is that it's a truly level playing field. Yes that also allows spammers to send outgoing shit smtp traffic, but I can deal with that.
Agree so much.
I really don't get the spam complaint anymore. Commercial providers like gmail do get a lot of spam (I have a gmail account as well, it gets more spam than ham) but it doesn't need to be that way.
I run all my own email infrastructure and spam is a non-issue. My primary email has been around very publicly since the mid-90s and is surely on every spam list ever but between a strict postfix configuration and a bayesian spam filter, spam is not a thing anymore.
I'm trying to think the last time I got a spam email in my inbox and can't remember. It's so rare.
While I mostly agree, I'm not sure it's possible to set up spam scoring properly. Either spam gets through, or some non-spam doesn't.
There's legitimate email that consistently gets marked as potential spam, and I don't know how to tell my mailreader to always trust that source. The tax service regularly mails me that there's a new message for me in my message box at the tax service site where I need to log in, which is a terrible way to communicate. Their paper mail does contain everything I need to know, but their email cannot because email is so insecure.
Something like email, but with authentication and encryption built in from the start, would be a really nice improvement.
I’ve never understood why I can’t optionally log in to my bank/hospital/tax site and upload my pgp key so that all communications from them are encrypted?
Inability, lack of willingness, lost ability to claim the customer screwed up, pick one
If anything they'd do S/MIME, but they often can't even get DKIM right. Maybe in the future.
Spam is not Email's fault, but the mere result of its success/importance, which has only been possible due to its interoperability, which in turn facilitates spam.
It would be the same if we had all been using - say- XMPP for the last 20 years.
Spam is totally email’s fault. They should have had disposable email addresses built into the protocol so they would be easy to support. Then when an address starts getting spam, run the spam filter more aggressively on emails to that address or block it entirely. Or they should have included PoW so it takes a little bit of CPU time to send an email (wouldn’t do anything about botnets, but not every spammer has a botnet handy.) I can’t think of any other ideas off the top of my head, but it’s absolutely true that there are ways that email could have been designed to be less of a problem
I mean, with the networks and processing speed of computers back when email was standardized, sending the email itself would be proof of work, no?
Less email as proof of work than having a known network address (this was when the hosts file was distributed manually, and bang paths were common) was proof of trust.
There were from a few dozens to a few hundreds of hosts on the entire network at this time. Most were educational institutions, meaning a campus sysadmin could run herd over users (if students) or refer to deans / departments (faculty). This from the 1970s through the early 1990s, at which point things began evolving rapidly.
Transacting email itself wasn't all that expensive, though for many sites it was batched via uucp or similar mechanisms.
Getting a PhD would have been the work required to get an email address so yes! Then eternal 09/ happened.
Proof of Stake as an accepted delegator to a university/government SMTP validator node.
Ultimately e-mail is a relic of a time when there was some level of trust (however small) among Internet users.
It's always going to be a bit of an awkward fit to today's Internet where you can trust that any and every exposed service will face some attempt at abuse.
Isn’t it just how you handle mailboxes. Let fred@domain.com accept and receive email at any fred+X@domain.com address with ability to block any specific address getting spam.
If the friends and family address gets spam, offer a way to create and distribute a new address.
For business it is tricker as you want anyone to email you. Here regular antispam measure are probably best.
DKIM, SPF, DMARC... old tech and it just got reactivated recently. Will take a time to spread.
On the other hand people might want to keep pseudonymous and throwaway mails. Nearly the exact same warnings apply here as they would have applied to those that chose Google as a mail provider and now complain about lacking support and account closure. A largely self-inflicted problem. Don't continue this naivete...
Spam is annoying, but not a threat. Phishing is to a large degree. That needs user training, people stopping to pay ransoms and sensible IT policies for corporations.
I suppose I agree for the narrower claim that a protocol which allows anyone to message anyone is going to inevitably have spam.
That said I can count on my fingers and toes the number of spam messages (via hijacked accounts) that have reached my Facebook Messenger/Instagram inboxes and have received zero spam in several years on Snapchat.
Walled gardens have many downsides, but there are upsides as well.
You don't get spam on Snapchat? I think you might define it differently than I do. Since those walled gardens have a verification step before allowing sending messages, spam on those platforms consists of "friend requests" from unknown people. If one is foolish enough to accept one of those, the more traditional spam message soon follows.
So yes, most walled gardens' two-step approach makes the spam more civilized, but they don't eliminate it.
Not a single one! I suppose I occasionally see friend requests from unknown users, but they don't actually convey a message or achieve any possible advertising goal.
That said, I only communicate with a couple dozen close friends on snapchat so perhaps a more active user will get spam.
Whether spam is or isn't email's fault, or a consequence of its architectural naivete, is arguable.
What's not arguable is that spam is email's problem.
What responses to my quite off-the-cuff though apparently resonating comment has (mostly) failed to recognise though is that spam is a problem not only for email users but across the spectrum for operators, administrators, senders, application designers, and a whole host of others. The fact, volume, and consequences of spam have a tremendous set of knock-on effects which are hard to express with any degree of coherence and sufficiency.
I get spam on Whatsapp, and FB Messenger. Not as much as email but I'm assuming because email is where the spammers are concentrating.
Which system gets it right and could email be fixed?
Really? I've never had spam on Whatsapp. I was just about to mention it as an example of a very successful service that didn't suffer from spam.
Interesting. It goes to show how varied people’s experience can be. I get zero spam on my seven year old Postfix server as long as I have greylisting enabled (my only anti-spam measure) while I get the occasional spam message on Whatsapp (that I only use for work purposes on my work phone since 2020).
I don’t use WhatsApp. I don’t have contacts on WhatsApp. I only registered a username to hold it. I only get spam on it.
WhatsApp doesn't have usernames?
Heh, you can tell how much I definitely don’t use it. I only recall when someone tried to get me to use mine they showed me a username (nickname?) and a QR code. So I must have registered a number at some point.
Most WA users I know get it, it's been a point of discussion in several group chats. But not enough you need spam filters, the need for which is the biggest problem with email because of false positives (I scan thru my junk mail folders weekly when I remember and find legitimate messages every time).
Undesired solicitations are going to be a factor on any sufficiently widely-used communications network. There are ways to reduce the impact, but most come down to some measure of real costs (as with postal mail), gatekeepers (as with, say broadcast media --- there's plenty of solicitation but it's at least curated, and no, that doesn't mean I can stand it myself), or systems with strong effective authentication and reputation capabilities, which would include most social networks.
It's trivial to address spam on trivial systems --- those which are small, centralised, and tightly controlled. It's difficult to do so on nontrivial systems --- those which are large, decentralised, and loosely controlled.
Email is very much the latter. It also lacks costs (sending has exceedingly low marginal cost), and has very poor authentication / identity assertion, and reputational capabilities. All of this is baked into SMTP at pretty low levels, and the various bolt-on kludges to address this (SPF, DKIM, DMARC, and anything else that I'm not yet aware of) are at best only partially effective, and rather fragile and fiddly. Among the reasons large monopoly email services are so effective and useful is that they "see" the spam problem at a global level, across hundreds of millions to billions of accounts. My statistical background tells me that this is almost certainly overkill, and that even services with only a few hundreds or thousands of widely-shared known addresses (something easily accomplished by honeypots) would achieve much of their effectiveness.[1]
Another huge problem for email is that there's a lot of loose coupling between end-user clients and servers, especially for desktop-based (non-Web or mobile App) systems. Contrast with social media or webmail in which a person's flagging of an item as spam or abuse is instantly registered by the system, which has full awareness of where the item originated and what other activity has come from that account (or if sufficiently sophisticated, known clusters of accounts) recently. In the case of email, the end-user client, the receiving server, the sending or relaying server, and the original injection point(s) might well be three or four entirely independent systems, for which there's no through chain of identity and often asynchronous hand-offs meaning that flagging actions are noted only long after the message was initially accepted for delivery.
That flexibility was useful early in the history of SMTP's development. It's an Achilles heel now.
One possible reform would be for sender to spool mail until it's been accepted for delivery. This would complicate sending (especially at high volumes, not necessarily a Bad Thing), but would mean that the determination of whether or not a message was spam, or behavioural assessments (e.g., Sender A has requested delivery to 1,000 local addresses, many of which don't exist and/or are honeypots) might permit a presumption of spam before the actual acceptance of the message.
All of which would raise costs to spammers and make use of botnets for delivery far less reliable in that those sending hosts would be identified as spammers before many messages could actually be transacted, assuming that white-hat recipients share reputation data regarding sending sources.
Another practice generally is to have a varying level of service provided based on the level of familiarity, trust, and/or value associated with specific senders. Given weak authentication this is not especially robust, but it would again make simple-minded email blast spamming highly ineffective. This practice has been fairly widely adopted in some forms by corporate domains which require specific whitelisting of authorised senders as a general rule. Implementations last I was aware tended to be ad hoc and kludgey. It's an annoying but reasonably effective method.
________________________________
Notes:
1. The law of large numbers and spam's reliance on broadcast distribution largely account for this. A small number of individual spammers hitting effectively all known email addresses account for a huge proportion of spam. Knocking a single such operation offline can drop global spam. This is a 2008 story in which a single provider accounted for 75% of all spam: <http://voices.washingtonpost.com/securityfix/2008/11/major_s...> . Smaller systems (< 100m accounts, say) might tend to miss more sophisticated targeted attacks and lag in responding to these --- phishing, spear phishing, and APT attempts against specific accounts. I'm not sure if this is a trade-off or not, though I suspect any sufficiently highly-place PEP (politically exposed persons) would attract such attention, and that this risk is not especially scale-responsive. This is also somewhat tangential to spam in the sense of indiscriminate mass mailings, though both are serious concerns for email integrity.
I more or less diagree with you, but the comment was lovely, and for some reason I find myself craving lobster thermidor.
<https://yewtu.be/watch?v=_bW4vEo1F4E>
This. I have been facing this issue recently more and more. Account creation emails aren't arriving to my custom domain recently. Some software companies seem to not send emails to custom domains. I use a well known commercial provider with DKIM and everything setup. Yet. I am beginning to use my custom domain only for human communication and all else gets directed into Gmail.
Huh—I wonder if that has to do with your TLD? I’ve been using custom domains for a few years and I have more issues with my .xyz domain than I do with my .com for both sending and receiving. Both have all the authentication DNS records set up.
I run my own email server with a custom spam filter that I wrote and one of the things I've done is blacklist certain TLDs that are so spammy that just this one mitigation cuts my spam by >50%. XYZ is one of those super-spammy TLDs, sorry.
Here is the complete list:
'top','tv','biz','rocks','ru','science','bid','date','casa','buzz','xyz','click','online'
I hate you and SpamAssassin. I also completely understand and will probably do the same when self hosting. Sad state of affairs.
No apology needed :) That's why I use the .com for everything serious and I don't have issues sending or receiving with that one.
I have zero issues with mail not arriving on my custom .me domain, and I’ve been using it for email for close to 10 years.
Do you have an example of sites where account creation mails do not arrive?
That yours has existed for 10 years might be helping you, you established some trust/history before the big providers stopped caring about sink holing false positives.
I have never heard of age being important for receiving mail. Sending, yes, as an annoying anti-spam measure, but why would anyone care about it the other way?
The second worst thing about email after spam, is all the solutions dedicated to fixing spam.
I've had to setup DKIM, update records and keep up with changes to it just to keep my custom domain.
> Some software companies seem to not send emails to custom domains.
As someone who operates a SaaS that does auto-block some account registrations based on email address, I feel like I should speak to this. We're actually fine with custom domains — and I imagine this is true for most SaaS companies. All our business customers use custom domains; and we're not about to blow holes in our funnel by making it harder to acquire new business customers. There's no logic in our codebase that especially favors the big email providers; no whitelist with gmail/outlook/etc. on it. (And we haven't found that any of the lifecycle-mail sending services we've tried have had trouble with custom domains, either.)
Instead, what we're mostly trying to stop is registrations using what you might call vanity or proxy domains — domains created by people who are only trying to obscure their real email address, but who aren't willing to invest the effort to make their domain into a "real" domain that has working email deliverability in both directions and maybe a web presence to go along with it.
Businesses almost always have such "real" domains; but people registering for illicit purposes (spamming, ban evasion, etc) almost never do, instead only bothering to set up e.g. an SMTP forwarding proxy that has no ability to originate mail. (Often they get such a service as a value add from their domain registrar; though there are independent services for this as well.)
To detect these, we check things like:
- Is your domain a known temporary email provider? (Immediate fail)
- Is your domain's MX record shared with one used by known temporary email providers? (Also an immediate fail; and zero false positives from this, surprisingly)
- Does your domain have SPF and DKIM records? (Good)
- Do your MX record hostname(s) resolve to the IP ranges of some skeezy VPS host, or to an IaaS's ephemeral-allocation IP ranges? (Bad)
- Does your NS record, or your MX record's NS record, point to a domain registrar that's well-known to be used for discovering + acquiring short nonsense domain names in bulk? (Bad)
- Does your domain have an apex A record? (Good) Is there a website there? (Good) Is that website a domain-parking page? (Very, very bad)
- How recently does your WHOIS record say that your domain was last transferred/acquired? Last 14 days? (Somewhat bad, but not as bad as the other things)
I would guess that if you're having problems, it's with one of these heuristics, or one in a similar vein.
If you jump through all the same hoops a business would jump through to make your custom domain "real", then any heuristics SaaS companies come up with to differentiate business domains from spam domains, should work out in your favor. (My own custom domain — which truly is just a vanity domain — passes all these tests easily; mostly by just doing the default stuff Google Workspace recommended I do when I set it up. Because that's all businesses are doing, too.)
Thank you for explaining how this works. Mine is a personal vanity domain with a non .com TLD. But I have the SPF and DKIM setup and I use a well known Google workspace alternative, which I have used for almost a decade now. So ideally I should have checked all the necessary boxes.
I hope, I have stumbled into a couple of services that have some poor email verification implementation and my domain has not been blacklisted by some email protection company or something serious like that.
> Account creation emails aren't arriving to my custom domain recently.
What do you mean by custom domain? Anything not @gmail.com? How could that be, since every corporation everywhere has a custom domain. Must be something else going on.
Consolidation of "trusted" mail providers (as if Google is trustworthy with its scanning of mails) is a real threat to federated mail. One can only hope that IT isn't stupid or greedy enough to sell it as that. To forbid the usage of Gmail should be sensible policy for any enterprise.
Spam is a problem, but honestly 15 year old slightly trained spam filter with a sensible config can filter all without any false positives.
Not that mail is not without its problems, but any modern alternative will suck much more. It will be a service attached to a corporation you will again make yourself dependent on. I feel for those whose Google accounts get locked, but to a large degree it is their own fault.
I don't know a lot of corps that block unknown mail servers. Have it sign outgoing mail with your known domains and your mail will not land in some sandbox or spam filter.
> To forbid the usage of Gmail should be sensible policy for any enterprise.
Instead, more than 60% of mid-sized companies use Gmail.
https://emailanalytics.com/gmail-statistics/
> 15 year old slightly trained spam filter with a sensible config can filter all without any false positives.
If it were that easy why do all the big players do it so badly? I'd rather get more spam in my inbox than miss the false positives (in fact most of what's in my spam folder is technically legitimate, but usually just broadcast emails from organizations I happen to have used my email address for at some point in the past). But I haven't found a way to sufficiently lower the threshold for spam detection for any email service I've used to avoid that problem.
It isn't easy, big players are doing very well percentage wise.
By contrast I've also had email for decades, and suffer effectively zero spam. Perhaps one difference is that I use two emails. One I use exclusively for messaging friends/family and the other I use for everything else. And I don't use the former email on any mobile device either.
The email I use for registering on sites, or using on mobile, is absolutely bombarded with spam. The email I use privately receives near zero spam (that isn't trivially filtered) even though it's a simple and common two word combination at a major mail server. Notably, I'm not registering for dodgy sites or anything like that on the spammed email. It's all major and ostensibly reputable sites. For sites I don't trust or care to receive a message from, I use throw-aways.
I simply assume any site you register on is going to sell, leak, or accidentally expose your email (or any other information you offer them). Similarly for mobile devices. And that seems to be a consistently validated assumption.
Somewhere in 1999 I started using unique email addresses for every single service or registration. For example, if I would register with Reddit today, then I would use 20220825_reddit_registration@example.com.
This makes it really easy to filter out spam (and I also know who sold my address). It's also trivial to block unwanted 'subscriptions'.
I use Thunderbird and create an identity for every one of these mails that require a reply (most don't) and TB automatically uses the unique address when replying.
The biggest problem of email is non verifiable routing due to header manipulation and lack of source authorization.
If these were in place then it would be able to send all spam back to sender, instead of putting it a spam folder due to filtering, your email client just would send it back to the origin address.
It also would help if each send would cost a token (of some monetary system) that would be returned on delivery at the legitimate destination. This would help against bulk sending.
Email is a system that will always cripple at some point if there aren't real world cost incentives in place.
People are the biggest problem with email, which is hard to solve with mere technical solutions.
If you don't like spam, postal mail is not any better. Worse, really, because there are no automated filters for it. Four out of five trips to the mailbox result in nothing but junk mail.
You forgot about the mess that is SPF, DKIM, and DMARC.
Spam, spam, spam, spam, spam, eggs, bacon, spam, and spam.
"but I don't like spam."
FWIW, I also omitted what's perhaps the most telling sign: increasingly it is not possible to find an email address for an entity or know whether or not a found address is useful or monitored.
That is: organisations are increasingly abandoning email, at least for public-facing work.
Ironically, Google are perhaps most notorious for this (HN is one of numerous Google-email-contact alternatives for problem resolution).
More generally, the end state of any universal communications medium of unlimited reach and access ... is abandonment. Absent some viable mechanism for curtailing usage, the only viable options become either shutting off the system entirely, or at best, randomly sampling from amongst submissions.
Conversation, and communication, both scale poorly.
I feel most tech organisations are now using Slack, Discord or Teams. At this point my work gmail is more an auth tool than an email.
Spam is not this big of an issue - filters for it are easily available and work well, even if you host your own.
https://news.ycombinator.com/item?id=32590389
Your response is essentially "yes it is". Apparently we disagree.
Re: spam - I have given up on my inbox altogether. I found that if there is a really important message that needs my attention, it tends to make its way through other channels (slack, voicemail, people just walking over to me etc).
Rest of it, is apparently not important to make a difference to me in any way.
postal services are awesome but tend to be forgotten. We need to use them more for personal, relevant things. Ideally handwritten.
The contemplative nature of reading paper is a bliss.
You were a trendsetter back then and are now, I guess.