SMTP server from scratch in Go – FSM, raw TCP, and buffer-oriented I/O
I’ve been using AWS SES for a while and realized I treated SMTP as a black box. To understand the protocol better, I built an MTA (Mail Transfer Agent) from scratch in Go. It handles the full SMTP lifecycle (RFC 5321) using raw TCP sockets instead of high-level frameworks.
The Engineering Challenges:
Finite State Machine (FSM): SMTP is strictly stateful. I implemented an FSM to enforce command sequencing (e.g., preventing DATA before MAIL FROM). It ensures that protocol violations are caught at the socket level with proper 503 Bad Sequence codes.
Buffer-Oriented Processing: Used bufio.Scanner to handle the byte stream. The biggest hurdle was the DATA phase logic—properly detecting the \r\n.\r\n sequence while managing memory efficiently using strings.Builder.
Concurrency: Leveraged Go's Accept() loop to spawn independent goroutines for each session, ensuring that the relay latency to Gmail (via STARTTLS) doesn't block the listener.
ISP Workarounds: Configured to run on port 2525 by default to bypass the common ISP block on port 25.
Status Codes Implemented: I implemented a subset of RFC 5321 codes, including 220 (Service Ready), 354 (Start Input), and error handling for 501 (Syntax) and 451 (Local Error).
Why I built this: Most modern tutorials stop at "How to send an email with a library." I wanted to see how the "dot-stuffing" mechanism worked and how a server actually negotiates a multi-step handshake over a raw connection.
I’d love to hear about edge cases I might have missed—specifically around handling malformed headers or managing long-lived TCP connections under load.
Source Code: https://github.com/Jyotishmoy12/SMTP_Server
Edge cases have probably slipped past my memory, but back in the day I iused to use qpsmtpd as an incoming SMTP server for processing a decent volume of mail.
As you'll see the core of qpsmtpd was very light but it was very extensible and allowed plugins to make changes/rejections to incoming mail at SMTP-time and that was very useful.
These days the project was superseded by https://haraka.github.io/
The only specific advice I'd have for you would be to use the SWAKS testing tool for various testing attempts in addition to the golang testing.
You should reject the obvious things (mail without Message-Id:, or Date: headers), you should avoid pipelining clients by dropping connections which start the transaction without waiting for the HELO/EHLO banner you send - which should be "slow".
You should probably also consider greylisting and similar built-in facilities. I found it very useful to automatically temp-fail 100% of incoming SMTP submissions if the system load were high, but that was back in the days when my SMTP server had 1gb of memory so it might be outdated.
Thank you for sharing such insightful knowledge. I appreciate it.