Why Email is Not Instantaneous — and Not Supposed to Be

The common perception is that email messages seem to arrive almost as soon as they are sent. Messages often appear to be delivered “instantaneously.” So, when an email delay occurs, it seems like something must be wrong. Sometimes there is a problem. Sometimes the delay is the result of normal email flow.

If the messages never show up at all, that is a different situation altogether. See “Where’s the Email? The Case of the Missing or Disappearing Email” for diagnosing those issues.

The multi-server delivery path

When an email message is sent, it is given to an email server for processing and delivery. That email server may forward it on to another email server, and so on, until it ultimately arrives in the recipient’s mail box.

Generally messages will pass though at least two email servers (the sender’s and the recipient’s). In some cases, when the sender and recipient are on the same machine and it is setup in a certain way, the email may be delivered on arrival. This is not unusual for internal corporate email, but rarely happens for general email messages.

In most cases, the sender’s and recipient’s email systems may each pass the message though multiple servers for various reasons, resulting in the message traversing many servers (a.k.a “hops”) in its delivery path.

Reliable delivery

Email is highly reliable – messages should always make it to their destinations, if at all possible.

Each server in the delivery path that accepts the email message is responsible for ensuring that the message makes it to the next server. If nothing goes wrong, the hand off can take place very quickly (in less than a second in many cases). However, many times, things do happen such as:

• DNS or network issues prevent the server from being able to determine what server is supposed to be next.
• Communications with the next server are temporarily failing due to network issues or internet congestion.
• The server itself is very busy at the moment.
• The next server is very busy at the moment and temporarily refusing connections.
• Additional processing of the message needs to take place before it can be relayed to the next server.

Each of these things can result in the message being delayed in reaching the next server.

If the server itself is busy or needs to perform further actions on a message, it may defer (or queue) processing of the message for a short time until it has available capacity. This often happens if there is a “spike” where many messages are arriving at a server in a short time, pushing its ability to process them all. In cases like this, servers respond by delaying some messages until they can catch up.

If the next server cannot be determined or reached or is not accepting email temporarily, then the message must be queued, and delivery retried later.

Queue processing delays

When messages are temporarily deferred for later retry or processing, they are often “queued.” This means that they are dumped into a special location for pending messages. Mail servers will check their queues periodically and process the messages waiting there to get them going. However:

• It is up to the mail server administrators to manage how often the queue is checked and processed. This interval can be low (like once/minute) or slower (like once every 5-10 minutes, every hour, or worse).
• If the queues are getting full (e.g. lots and lots of messages are there), processing them all may take a long time.
• Even if a message is quickly retried, it will not be delivered until the next server is available and reachable.

Generally, messages are kept in mail queues and retried over and over for up to five days; however, some systems (such as public providers like AOL, Hotmail, Yahoo, Gmail, bulk mailers, etc.) may have much shorter grace periods for successful delivery.

Size delays:

Some other common reasons for apparent delays include:

• Large Messages: Messages size can affect delivery times when it is transmitted over relatively slow, low-bandwidth, or very busy connections. It may take some time to upload a large message to the server, resulting in an apparent delay. For example, if you have a 50MB email message to send and a 512 Kbps DSL line, it will take over 13 minutes to upload the message. Additionally, large messages will slow down processing and delivery at every stage in the process.
• Many Recipients: Messages addressed to large numbers of recipients take much more work to process, resulting in additional apparent delays.

“Sender Offline” delays:

Often when customers ask about message delays and we look into the cause, it appears that for example the message was sent last night but did not arrive until the following day. What has often occurred in these cases is that the sender composed and sent the message when offline. Their internet connection was down, they already unplugged, etc.  The “Date” stamp on the message is when they “sent” it (and it was put in their Outbox). However, the message never left the sender’s computer until the next day when the sender went back online and opened his/her email program.

In this case, the apparent delay is the sender’s “fault” for not being online when sending.

Occasional delays aren’t uncommon

When multiple servers must each process the message, must be able to communicate with each other, and must have the capacity to manage the processing requirements, delays can and do occur. In fact, the more servers in the path, the more likely a delay is to happen.

It is typical to see occasional delays of up to three minutes in email delivery due to random issues like server processing loads and network traffic spiking here and there on the internet.

Delays of tens of minutes to hours are most often the result server maintenance or outage issues, or mail queues not being properly or promptly processed.

Where was the delay?

We are commonly asked to look at a message that was received to determine where/why the delay occurred. It is relatively easy to figure this out.

First, you need to get the full headers of the received email message. See Viewing the Full Source/Headers of an Email message.

An example delayed email messages might contain header lines that look like this:

Received: via dmail for +INBOX;
        Tue, 3 Feb 2013 19:29:12 -0600 (CST)
Received: from abc.luxsci.com ([10.10.10.10])
	by xyz.luxsci.com (8.13.7/8.13.7) with
        ESMTP id n141TCa7022588
	for <user-999@xyz.luxsci.com>;
        Tue, 3 Feb 2013 19:29:12 -0600
Return-Path: <test@sender.com>
Received: from [192.168.0.3] (verizon.net [44.44.44.44])
   (user=test@sender.com mech=PLAIN bits=2)
   by abc.luxsci.com (8.13.7/8.13.7) with
   ESMTP id n141SAfo021855
   (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA
   bits=256 verify=NOT) for <test@domain.com>;
   Tue, 3 Feb 2013 19:18:05 -0600
Message-ID: <4988EF2D.40804@domain.com>
Date: Tue, 03 Feb 2013 20:10:10 -0500
From: "Test Sender" <test@sender.com>
MIME-Version: 1.0
To: "Test Recipient" <test@domain.com>
Subject: Example Message
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-Comment: Lux Scientiae SMTP Processor Message ID -
   1233710941-9110394.93984519

It is the “Date” and “Received” header lines that are of importance.

The “Date” header is added by the sending email program (i.e. Outlook) and may be blatantly inaccurate, for example if the sender computer’s clock is off. Also, the “Date” is added when the message is sent, not when the message leaves the sender’s “Outbox.” So, if the sender is offline for some reason, or the message is otherwise sitting in the Outbox, the message may look “delayed.”In this example, the delay is merely in getting the message off the sender’s computer.

The “Received” header lines are added by the mail servers that process the message. One Received header line is added each time a mail server accepts the message for processing. They are added from the “bottom up” — so the last added Received line is at the top and the first is at the bottom.

You can detect where the delays happened by looking at these header lines, in order, and comparing the date and time stamps (once you correct for differences in time zones).

In this contrived example, we see that:

1. The message was sent at 8:10:10 pm Eastern Time
2. The message was delayed in the sender’s Outbox for 7 minutes, 55 seconds, based on the first “Received” line. It also may have taken a long time for the message to be uploaded to the server, i.e. if the message was big and the connection was slow.
3. For some reason, this mail server delayed the message for about 11 minutes before it made it to the recipient’s server, where it was accepted and delivered immediately.

One could then ask the IT staff running example server “abc.luxsci.com” to look into their logs for information as to why the message was delayed for 11 minutes, if this delay is significant to you.