Where’s the Email? Diagnosing and Resolving Issues with Missing Email

In many ways, the Internet is still like the Wild Wild West. Email messages sent to you or from you can and do “go missing” for no apparent reason.  This can happen no matter what email provider you use. So, what happened to these “AWOL” messages?  How can you diagnose and solve the problem?

In some cases, you can figure out the reason yourself.  In most cases, you may need the assistance of your email provider’s Technical Support team — they should be experts at this kind of “Sherlock Holmes” work because it is so prevalent… we receive questions about missing messages on a daily basis.

However, many email providers’ (we have heard reports of unsolved email mysteries at popular freebie email providers that offer paid business accounts) Technical Support teams will not assist you with these kinds of issues.  Their idea of a resolution might be “your email is gone … have it resent”.  That can be frustrating to you, the customer;  however, what it comes down to is that their Tech Support may not want to “waste” time with matters like this because their margins and prices are so low.

At LuxSci, we have never lost a client because we have lost their email (to our knowledge). Instead, very often we help our clients “find” missing messages or determine what might have happened to a message.   Excellent customer support is a standard feature of our services.  It brings to mind the old adage “you get what you pay for”.  If email delivery is not a critical success factor of your business, then you may be able to justify your email provider’s unwillingness to troubleshoot technical issues.  But where do you turn when email you need, or expect, or don’t even know about, never gets to your Inbox?

In this article, we will discuss a number of the things that can cause email messages to go missing and appear to never be delivered. As the possible problems are really limitless, we will discuss some of the most common ones.  Any particular issue may require extensive diagnosis, knowledge of how email flows between servers, how the servers involved behave, knowledge of the content of the message sent, etc.  At the end of this article, we will go over how an email provider might actually track an email message through its systems.

Some Messages Really do “Just Disappear”

There are many things that can cause an email message to simply disappear, for real, and be forever lost in the void.  Messages that “just disappear” are ones that are not received by the recipient and for which the sender never receives a “bounce message” indicating why or even that the delivery failed.  Some of the reasons for missing email include:

Incorrect “From” Address

Many email servers, including LuxSci’s, will do some basic validation on the “From” address used in an email message for anti-spam and anti-fraud reasons; if it is determined to be invalid, the message will be denied/bounced.  However, since the “From” address is invalid, the bounce never makes it back to anyone and the message seemingly disappears.  What can cause an invalid “From” address?

1. The sender misspelled  his/her email address when setting up his/her email program.
2. The sender’s domain name does not exist or its registration has expired.
3. The DNS on the sender’s domain has been changed and now has no DNS MX records.
4. The sender’s domain is or was hosted by the same email provider that the recipient uses; as far as that provider knows, your specific “From” address is not valid on their system although the your domain is apparently still active there.

What can you, as a sender, do to diagnose this?

1. Look in your “Sent” email folder and find the “From” address. Check for subtle spelling errors and check to be sure that your domain’s registration and DNS are active (Check EasyWHOIS and the MX Toolbox).
2. Have the recipient contact their email provider and ask that they look in their server logs for any message from you that may have been bounced or denied. You will need to provide the “From” address used, the recipient’s address, and the time/date that the message was sent.

Filters, Filters, Filters

If the recipient of the message has any kind of email filtering (Spam filtering, Virus filtering, Content filtering, etc.) it may be that the message was inadvertently filtered and saved in some folder or quarantine area, discarded, or bounced.  It is also often the case that email providers filter certain kinds of messages without the recipient’s knowledge — to provide “a better service”.  What to do?

1. If the recipient has a place s/he can go to look for spam or quarantined email, that is the first place to check.
2. If the recipient has any custom filters setup on his/her email server or email program, s/he should check those and see if any of them may have inadvertently caught the message in question.
3. The recipient should check with his/her email provider to see if  the message was filtered by some server-side rules that s/he has no control over.  The provider may need to look in its mail server logs to check.  You, the sender, will need to provide the “From” address used, the recipient’s address, and the time/date that the message was sent.

It Was Deleted

It is very often the case that the recipient actually did receive the message, or that it was at least delivered.  S/he might not be able to find the message because:

1. The recipient accidentally deleted the message.  This happens.  Perhaps there is a backup or archival area where the recipient can look for another copy of the message.
2. The recipient has some custom email filters that caused the message to be saved in a different email folder than expected (or caused it to be forwarded or otherwise diverted).
3. The recipient could be checking his/her email using multiple email programs.  One program could have downloaded and deleted the message, thus making it appear in the other programs like it never arrived.  This is common when people use a combination of desktop programs, like Outlook or Thunderbird, and mobile devices, and have the mobile devices configured to delete email from the server.
4. The recipient is looking in a Web-based email interface and his/her POP email program, i.e. Outlook or Thunderbird, at work or at home has already downloaded and deleted the message from the server.  This is the “Oops, I left my computer logged in!” scenario.

The recipient’s email provider can look in their email logs to see if the email message was ever actually received by them and, possibly, what happened to it after it was received.

Server, Firewall, or Blacklist Trouble

Some email server at the sender’s provider, the recipient’s provider, or somewhere in between could be having technical difficulties and either

1. have accidentally deleted the message,
2. have it queued somewhere from where it is not being released, or
3. have trouble sending it onward due to network connectivity issues, firewalls, blacklisting, etc.

This situation is more difficult to diagnose. You will need to start with the sender’s email provider and have them track the message sent as far as they can (see the end of this article for an idea of how this is done). They will be able to determine if there was any problem on their side, or if not, will provide details as to where it was delivered and tracking codes that they can use to further extend the search. Proceeding through each provider will eventually yield the culprit … or a service provider who doesn’t care and won’t help you.

Messages that Bounce: Temporarily or Permanently

“Bouncing” is another term for a message that is “denied” by a server and returned to the sender.

Messages may seem to disappear or never be delivered because, in fact, they are “bounced” or “denied”.  There are two kinds of message delivery failures: temporary and permanent.

Temporary Failures: 400 errors

A temporary failure, a failure where the target mail server returns an SMTP error code between 400 and 499 (or where the target mail server is down and not accepting connections), means that that mail server is temporarily refusing to accept the email message, but might accept it later.  There are all kinds of reasons why this could be: maybe it’s too busy, maybe the recipient is over his/her disk quota, etc. When one mail server tries to talk to another and gets a “temporary failure” like this, the mail server will queue the message and try again later.  Usually, mail servers will try for up to 5 days before giving up; after 4 hours, they usually send a notice back to the sender explaining why there is a delay.  At the end of 5 days, a final delivery failure message is sent back to the sender.

Note: not all providers send the 4-hour explanation message.  Also, many free, bulk, and public email providers will not keep trying for a full 5 days, but only for a few hours so as to conserve resources.

So, a missing message could be stuck in transit, in the process of “trying again and again” for many days; and… you may or may not get any indication of this.  The way to diagnose this is the same as with the “Server, Firewall, or Blacklist Trouble” problem — contact your email provider’s support team and ask them to start the tracking process.

Permanent Failures: 500 errors

A permanent failure, a failure where the target mail server actively returns and SMTP error code between 500 and 599, means that the mail server will not accept the delivery of this message and it should be returned to sender (and not re-tried).  Some typical reasons for these kinds of refusals include:

1. the recipient’s email address is defunct or doesn’t exist
2. the email server no longer hosts email fro this recipient address — e.g. their account has been suspended or closed, or the customer ha moved to a new provider without properly updating the domain’s MX records
3. the sender misspelled the recipient’s email address
4. the recipient’s spam filters have denied the message
5. the sender’s mail server is blacklisted
6. the recipient’s mail servers are misconfigured
7. the recipient’s domain name has expired or his/her DNS MX records are incorrect
8. etc., etc.

Permanent errors generate a delivery failure message that is sent back to the sender.

Non-delivery Reports / Delivery Failure Messages

Messages sent back to the sender to report the failure of delivery to a recipient are called “non-delivery reports” (NDR).  These usually indicate exactly why the message was not able to be delivered.  If you have trouble interpreting one of these, ask your email service provider’s support team … they should be able to decipher it for you, as they deal with these as a matter of course.

Any problems with missing email are most easily solved if the sender gets a bounce message / non-delivery report.  If you are missing a message that someone sent to you, ask them if they got a bounce.  This may resolve the problem straight away; maybe they just misspelled your email address!

The Sender Never Received a Bounce?

It is possible that a non-delivery report was sent back to the sender, but s/he never got it!    This could be for all the same reasons that email might go missing in the first place, as well as:

1. the sender could be filtering out bounce messages for anti-spam reasons, and thus doesn’t see legitimate ones.  This is common if someone is using the sender’s email address to send spam and the sender is trying to get rid of all the back scatter non-delivery reports.
2. the sender’s From address could have been invalid or misspelled, so the bounce message never makes it to him/her.
3. the sender’s spam filters could be catching the bounce message.
4. the recipient’s mail servers may be configured to handle email for the sender’s domain (maybe because the sender used to host email there), and thus the bounce message may have been delivered locally there and not to the servers where the sender really gets his/her mail.

Other Strangeness that Affects Message Delivery

There are many other things that can affect the apparent deliverability of an email message.  Some of these include:

1. The message could still be in the sender’s “Outbox” because the sender’s computer is not connected to the Internet, or because his computer is having issues with the sending of email messages.
2. The sender may have “sent” the message but turned off his/her computer before the message actually left his/her Outbox .. so the recipient would not receive the message until the sender goes back online, perhaps the next day (this is a big source of “delayed” email issues that we see).
3. Someone else logged into the recipient’s email account and deleted the message before s/he saw it.
4. The message was too big and got stuck or refused somewhere along the way and a bounce couldn’t get back, as the bounce was trying to include the entire message.
5. The sender’s or recipient’s email servers could be having DNS issues causing them to think that the sender’s or recipient’s domain name is non-existent or has no MX records.  This could result in the message bouncing and the bounce failing to be delivered anywhere.
6. A server crash somewhere could have caused a loss of data which included the contents of the message-in-transit.

What Can you Do To Mitigate Message Loss?

There are several things that can be done:

1. Make sure that your email provider’s Technical Support team is available and willing to help you track down any issues that occur.  The cause might not be theirs, but their assistance may be needed to determine where the real problem lies.
2. Use a “backup” system where copies of your recent email are stored in a separate folder, location, or archive.  In the case that your message is somehow inadvertently deleted after delivery to you, you will still have a way of retrieving a copy of it.  LuxSci provides both simple “Backup” folders that save copies of your most recent email, and archival systems for keeping copies of all sent and received messages.
3. Understand your email filtering rules.  Know what happens to different kinds of messages and what kinds of filters are in place.   Ask your provider for help in getting a summary of these rules.
4. Ask your provider what system-wide rules are in place that may affect email delivered to you.  Find out what happens to messages affected by these rules.
5. Choose a provider that provides delivery status reporting.  This way you can easily see if messages that your send actually make it to and are accepted by your recipient’s email system.  E.g. LuxSci provides this.

It comes down to (a) having able and willing support, (b) having reliable backups, and (c) educating yourself about your own filters.  This is the best you can do and will go a long way to ensure that when you discover a missing message, that you are able to recover it or at least find out what happened.

It should not be a “mystery” what happens to missing messages.

How an Email Provider Tracks a Missing Message

Each server at an email provider may handle hundreds of thousands or millions of email messages in a single day.  The log files saved to track what happened with these messages can be hundreds of megabytes or even gigabytes in size for each day.  For this reason, email providers  (a) only keep their logs for a short period of time, and (b) cannot search for your missing mail without some good details to go on.

1. If you are missing a message, ask for help sooner rather than later.  The longer you wait, especially if you wait for weeks, the more likely it is that the mail provider no longer has the logs necessary to track what happened.  For example, LuxSci only keeps these mail logs for two weeks…. unless a customer has a dedicated server and a special need for longer retention.
2. Tell your mail provider what server you sent the messages through.  You may have options like: SMTP servers, WebMail servers, etc.
3. Give the technical support team as much information as possible to help them comb their logs.  Typically, they will want
   1. The date and time that the message was sent (be sure to let them know what timezone this information is in).
   2. Exactly who the message is from.
   3. Exactly who the message is to.

   The easiest way to get this information is to copy the full headers of the message from your “Sent” mail folder. In this way, you will have the exact details, including any subtle spelling or formatting errors that may be present.

Next, your mail provider will look in their mail server logs for messages that match the criteria provided.  If they do find the message, the message will have been assigned an “ID code” by their servers.  They will be able to use the is “ID Code” to track the message on their servers and see where it went.

These mail logs lines will indicate if there were any errors or reasons why the message could not be processed or sent.  If all was well, the log lines will record how their servers talked to the recipient’s servers and if the message was accepted.  There are several possibilities here:

1. Messages permanently denied.  This is the “permanent error” discussed earlier which generates a bounce back to the sender.  Your support team will see a record of this along with the reason provided by the recipient’s mail server.
2. Messages temporarily denied.  This is the “temporary error” discussed above which causes the message to be queued to keep trying to send again and again.  Your support team will see a record of this along with the reason provided by the recipient’s mail server.  They should also perhaps see the messages still in the mail queue.
3. Your mail provider cannot connect to the recipient’s servers due to network connectivity issues, firewalls, or blacklists.  This will be recorded in the mail logs and usually treated as another kind of “temporary failure”.
4. The messages are delivered to and accepted by the recipient’s mail server.

In case #4, the recipient’s mail server generally tells what “ID Code” it is using to identify the message in its system and what server accepted the message.  At this point, your mail provider has done its job and delivered the message out of its systems.  Any remaining delivery problems lie with the recipient’s system.  The next step would be to take the “ID Code” obtained from the logs and give this to the support team at the recipient’s mail provider and ask them to see what happened to that message on their side.  With the “ID Code” and name of the server that accepted the message, the recipient’s support team should not have any problem continuing the tracking process.