How to Set Up an SMTP Server: A Step-by-Step Guide
Prepare server
sudo hostnamectl set-hostname example.comsudo apt install mailutils postfix- Choose "internet site", and type your domain (example.com)
Test postfix
mail your-test@yopmail.com -s "Subject"- You should receive a mail from
debian@example.com
TLS
Generating certificates
sudo apt install certbotsudo certbot certonly --standalone --rsa-key-size 4096 --agree-tos --preferred-challenges http -d example.com- You might need to kill your running webserver (port 80) to complete the challenge in standalone mode
Giving rights to postfix
sudo chown -R root:postfix /etc/letsencrypt/live/example.com
Postfix config
/etc/postfix/main.cf
# TLS parameters
smtpd_tls_cert_file=/etc/letsencrypt/live/example.com/fullchain.pem
smtpd_tls_key_file=/etc/letsencrypt/live/example.com/privkey.pem
smtp_use_tls=yes
SPF and DMARC
- SPF:
v=spf1 ip4:<your_ipv4> ~all - DMARC:
v=DMARC1;p=none;pct=100;rua=yourmail@example.com;sp=none;aspf=r;
DKIM
- https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-dkim-with-postfix-on-debian-wheezy
- All the check-auth tests should pass (SPF, DKIM, DMARC)
SMTPD
Enable submission
- Uncomment submission's lines in
/etc/postfix/master.cf - change
smtpd_sasl_type=dovecotbysmtpd_sasl_type=cyrus - add
-o smtpd_sasl_security_options=noanonymous
Install Cyrus SASL
sudo apt install sasl2-binsudo usermod -aG sasl postfix
Configure SASL
/etc/default/saslauthd
START=yes
MECHANISMS="sasldb"
OPTIONS="-c -m /var/spool/postfix/var/run/saslauthd"
sudo systemctl restart saslauthdsystemctl status saslauthd- Make sure it is running with the
/var/spoolarguments
- Make sure it is running with the
Add a user
sudo saslpasswd2 -c -u example.com user- The user does not have to be the user you will send mail with, it is just credentials
sudo testsaslauthd -u user -p password -f /var/spool/postfix/var/run/saslauthd/mux- Add a space before the command to not keep this in your history
- Note the custom socket path, it is required
Link postfix with Cyrus
/etc/postfix/sasl/smtpd.conf
pwcheck_method: saslauthd
mech_list: PLAIN LOGIN
sudo postfix reload
Final test
- Use https://www.mail-tester.com/ to make sure everything is working properly
- Example code (TS) below, can be ran with
npx ts-node test.ts - Use real data to test your setup or SpamAssassin will not be happy
import nodemailer from 'nodemailer';
const smtpEndpoint = 'example.com';
const port = 587;
const senderAddress = 'My name <my-address@example.com>';
const smtpUsername = 'user';
const smtpPassword = 'pass';
const transport = nodemailer.createTransport({
host: smtpEndpoint,
port: port,
secure: false,
auth: { user: smtpUsername, pass: smtpPassword }
});
transport.sendMail({
from: senderAddress,
to: 'your-mail-tester-addr',
subject: 'A real subject',
text: 'A real body'
});
Additional steps for Outlook/Live
Configure virtual aliases
/etc/postfix/main.cf
virtual_alias_maps = hash:/etc/postfix/virtual
/etc/postfix/virtual
@example.com debian
sudo postmap /etc/postfix/virtual
Sign up to sendersupport
- https://sendersupport.olc.protection.outlook.com/snds/index.aspx
- Claim your IP with abuse@example.com
- Open your mail on the server with
mailand confirm IP's ownership
Useful to debug
tail -f /var/log/mail.info-vin submission parameters (/etc/postfix/master.cf)
Additional resources
- http://www.postfix.org/SASL_README.html#server_cyrus
- https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-postfix-as-a-send-only-smtp-server-on-ubuntu-18-04
Before You Run Your Own SMTP Server
Running an SMTP server is possible, but it is not the easiest path to reliable inbox placement. Mailbox providers judge your domain, IP address, authentication, sending history, complaint rate, bounce rate, and message quality together. A technically working Postfix server can still land in spam if reputation is weak.
Use your own SMTP server when you need control and understand the operational work. For ordinary newsletters, transactional mail, or cold outreach, a reputable email service provider is usually easier to maintain.
If you do self-host, start with these assumptions:
- You need a dedicated IP with clean reputation.
- You need reverse DNS that matches your sending hostname.
- You need SPF, DKIM, and DMARC configured before sending volume.
- You need TLS certificates and submission authentication.
- You need bounce handling and suppression.
- You need logs, monitoring, and rate limits.
- You need a warm-up plan before sending real volume.
Skipping any of these can make the server appear suspicious even when your mail is legitimate.
DNS Records You Should Confirm
DNS is where most self-hosted SMTP setups fail. At minimum, confirm these records before testing campaigns.
A Record
Your sending hostname should resolve to the server IP.
mail.example.com -> 203.0.113.10
PTR / Reverse DNS
The IP address should resolve back to your hostname. This is configured with your hosting provider, not inside your domain DNS panel.
203.0.113.10 -> mail.example.com
MX Record
If the same server receives mail, the domain needs an MX record. If it only sends mail, an MX record is still useful for abuse, bounce, and postmaster addresses.
example.com MX 10 mail.example.com
SPF
SPF authorizes your server to send for the domain.
v=spf1 ip4:203.0.113.10 -all
Use ~all while testing if needed, but move toward -all once every legitimate sending source is accounted for.
DKIM
DKIM signs outgoing mail. Without DKIM, forwarded mail and modern filtering can become harder to pass. Use a current key length and rotate keys periodically.
DMARC
Start with monitoring, review reports, then move toward enforcement.
v=DMARC1; p=none; rua=mailto:dmarc@example.com
When you understand all legitimate sources, move gradually to p=quarantine or p=reject.
Deliverability Checks After Setup
Once Postfix can send, do not jump straight into campaigns. Test the server like a production system.
- Send to a mailbox you control at Gmail, Outlook, Yahoo, and your own domain.
- Inspect message headers to confirm SPF, DKIM, and DMARC pass.
- Confirm TLS is used.
- Check that the
From,Return-Path, and DKIM signing domain align. - Send a realistic plain-text message and a realistic HTML message.
- Watch
/var/log/mail.infofor deferrals, rejects, and authentication errors. - Add hard bounces to a suppression list immediately.
You should also set up role addresses such as postmaster@example.com and abuse@example.com. Some providers expect them to exist, and they help with reputation and incident handling.
Warm-Up and Sending Limits
A new SMTP server has no sending reputation. Even if authentication is perfect, high volume from a new IP or domain can trigger filtering.
Start small:
- Day 1-3: send only a small number of messages to people who expect them.
- Week 1: monitor bounces, complaints, and replies.
- Week 2: increase gradually only if results are clean.
- Ongoing: keep bounce rate low and avoid sudden spikes.
Do not warm up with unverified or purchased lists. Use email verification before any campaign so hard bounces are removed before they hit your reputation.
Security Basics
An SMTP server is a target. If it becomes an open relay or a weakly protected submission server, it can be abused quickly.
Use these safeguards:
- Require authentication on submission port 587.
- Disable open relay behavior.
- Use strong passwords or application-specific credentials.
- Keep Postfix, SASL, and system packages updated.
- Restrict admin access with SSH keys.
- Monitor queue size for unusual spikes.
- Rate-limit accounts that send unexpectedly.
- Keep backups of configuration files before major changes.
If your queue suddenly fills with mail you did not send, stop Postfix, investigate credentials, rotate passwords, and check logs before bringing it back online.
When to Use a Sending Provider Instead
Self-hosted SMTP is educational and powerful, but it creates maintenance work. Use a provider when you need:
- Shared or managed IP reputation
- Webhooks for bounces and complaints
- Suppression-list automation
- Analytics and event tracking
- Easier DKIM setup
- Support during deliverability incidents
Use your own server when control matters more than convenience. Either way, the fundamentals stay the same: authenticate the domain, verify the list, send wanted mail, monitor results, and remove bad addresses quickly.
Common Troubleshooting Scenarios
If mail sends but does not arrive, start with logs. Postfix usually tells you whether a message was accepted, deferred, rejected, or stuck in the queue.
Useful commands:
mailq
sudo postqueue -p
sudo tail -f /var/log/mail.info
sudo postfix check
If messages are deferred, read the remote server response. A DNS problem, blocked port, missing PTR record, or poor IP reputation will usually appear in the log message.
If authentication fails, inspect the final message headers in a mailbox you control. Look for spf=pass, dkim=pass, and dmarc=pass. If DKIM fails, check the selector, DNS TXT record, private key path, and whether the signing service is running.
If Gmail or Outlook accepts mail but places it in spam, review reputation and content. Authentication passing is only the baseline. You still need consistent sending volume, low bounces, clean links, and recipients who actually want the message.
If bounces increase after importing a list, stop and verify the list before sending again. SMTP configuration cannot compensate for stale or low-quality data.
