E-mail Troubleshooting - The Mail Must Get Through!
By Gideon T. Rasmussen - CISSP, CISM, CFSO, SCSA

There is an old expression, through rain or sleet or dark of night, the mail must get through. The same sense of urgency applies to the delivery of e-mail. This article details how e-mail flows between mail servers, through firewalls and across the Internet. E-mail can be difficult to troubleshoot because it uses SMTP (Simple Mail Transfer Protocol), DNS (Domain Name System) and TCP/IP (Transmission Control Protocol/Internet Protocol). To troubleshoot e-mail outages, start with DNS troubleshooting and consider the basic concepts of network troubleshooting as well.

HOW E-MAIL TRAFFIC FLOWS

An originating mail server must resolve the name of the destination mail server in order to send e-mail to it. If no caching is involved, the process begins by determining the DNS server of the destination domain. From there the authoritative DNS server is queried to determine the preferred mail server and resolve its IP address. Once this process is complete, the mail server sends the message using TCP/IP.

DNS will either provide the public IP address of the destination mail server or the external interface of a firewall. In place of public addresses, most sites enable dynamic Network Address Translation (NAT) to obscure the private IP addresses of internal systems. In this case, the firewall must provide a static NAT or a proxy to pass the SMTP traffic to the private address of the mail server. When DNS resolves to the static NAT of the mail server, a permit rule or the SMTP proxy "Inbound Through" configuration must be used. When DNS resolves to the external address of the firewall, the "Inbound To" setting must be used and the SMTP proxy passes the traffic to the mail server.

TROUBLESHOOTING

Inbound Troubleshooting

  1. From an external system, use the nslookup command to verify that DNS resolves the IP address of the mail server (set type=SOA). Is the IP address of the mail server what you expect? If the IP address is incorrect, inbound mail will never reach the server.
  2. Are the correct combinations of configurations in place?:
    a. If the SMTP proxy "Inbound Through" setting is in use, a static NAT or public address is required.
    b. If the SMTP proxy "Inbound To" setting is in use, the mail server should be entered in the proxy's servers tab.
    c. If a SMTP permit rule is in use, a static NAT or public address is required.
  3. If a static NAT is in use, does it point to the private address of the mail server?
  4. Determine if mail traffic is reaching the firewall. On a CyberGuard firewall, enter "netguard -An | grep mail server IP address". A "P" in the second column indicates the traffic was permitted. If a proxy passes the traffic, an "Xr" will be present. If you see a "D" there, the traffic was denied. If you don't see anything, mail is not currently flowing through the firewall or the traffic never reached the firewall.
  5. Try using the ping command to reach the mail server's IP address (this requires an ICMP permit rule). If you are unable to ping the mail server, the issue pertains to the network or the availability of the mail server.
  6. Try to telnet from the firewall to the mail server (requires a SMTP permit rule):
    NETWORK > telnet mail.cyberguard.com 25
    helo mail.example.com
    mail from: jsmith@example.com
    rcpt to: jsmith@cyberguard.com
    data
    Subject: Test message

    Did you get this?
    .
    quit
    NOTE: The "." dot on a line by itself sends the mail (above quit).
  7. Check the firewall and mail server for error messages.
  8. Capture SMTP traffic flow with the tcpdump command. Examine the output with Ethereal.

OUTBOUND TROUBLESHOOTING

  1. From the mail server, use the nslookup command to resolve a domain's mail server (set type=SOA). If this fails, troubleshoot the issue until DNS functionality is restored. The mail server must be able to resolve names in order to send mail.
  2. From the mail server, use the ping command to try to reach its default gateway.
  3. Review the mail server and firewall error logs.
  4. Determine if mail traffic is reaching the firewall. On a CyberGuard firewall, enter "netguard -An | grep mail server IP address". If a proxy passes the traffic, an "X" will be present.
  5. Capture SMTP traffic flow with the tcpdump command. Examine the output with Ethereal (See previous article "How Network Traffic Flows").

TROUBLESHOOTING E-MAIL ALERTS

In order for mail alerts to be sent from a firewall, DNS must resolve the mail server to its private IP address. The best method to facilitate this is to use the Split DNS configuration. Otherwise, DNS will resolve to the public IP address of the mail server, the firewall will send alerts out via its external interface and delivery will fail. A permit rule from the firewall to the mail server must also exist.

One way to see if mail alerts are leaving the firewall is to generate an alert and check /var/spool/mailq/outgoing/smtp and /var/spool/mailq/data. If these directories contain files, the e-mail alert did not go out. If there are no files present, the alert was sent successfully.

Troubleshooting e-mail can at times seem a difficult task. Start with the basics of DNS resolution and network connectivity and work from there. The root cause rarely lies with the SMTP protocol.

How Network Traffic Flows - Getting Started

Network Troubleshooting - A Complex Process Made Simple

DNS Troubleshooting - Everything Depends On It









Copyright © 2005 CyberGuard Corporation All Rights Reserved.
Reprinted with Permission