Customers of our Secure High Volume bulk outbound email service often ask how they can “send faster.” They want to get their mailing out ASAP, no matter if it is to hundreds or millions of recipients.
This post codifies all of the various types of advice we give for optimizing outbound email throughput. Much of it applies to outbound email sending over SMTP in general — i.e. its not limited to the LuxSci Secure High Volume service.
Use Concurrent Connections
When sending an email message, your emailing program connects to our servers, establishes your identity, and passes the message through. If you have to send 1000 messages, then you might connect 1000 times to do this. Many sending programs can be configured to make more than one connection at a time. If you make 10 connections at once (e.g. concurrently), then you could send those messages about 10 times faster. That is a really significant speedup!
Don’t make too many concurrent connections, however! The more connections that you make at once, the harder your server has to work to process the mail. At some point, the server can get so busy and overloaded that the average time to send a message starts getting longer and longer. You never want to push your server to the point where it is struggling to keep up with your sending, as that will only make things slower for you. Instead, you want to use a modest number of concurrent connections so you can take advantage of parallel sending and so the server can easily and efficiently process all of the messages.
For shared High Volume accounts, where you are sharing capacity with other bulk senders, we recommend keeping your number of concurrent connections to 1o or less. Single dedicated servers can support 20-30 concurrent connections (or more depending on many factors discussed below), and dedicated server clusters can support as many as you need (depending on how large a cluster you have).
SMTP Pipelining
Your email program sends a message by:
- Connecting to your SMTP server
- Establishing SSL or TLS encryption, if configured
- Authentication to establish your identity and permission to send
- Uploading the list of recipients and message content
- Disconnecting
When sending small messages, the time taken by steps 1, 2, 3, and 5 are very significant relative to the time it takes to upload the message data. With SMTP Pipelining, the connection is reused for successive messages. I.e. when sending 3 messages, it would look like:
- Connecting to your SMTP server
- Establishing SSL or TLS encryption, if configured
- Authentication to establish your identity and permission to send
- Message 1: Upload the list of recipients and message content
- Message 2: Upload the list of recipients and message content
- Message 3: Upload the list of recipients and message content
- Disconnecting
By not repeating the connect-authenticate-disconnect steps for every single message, a lot of time in saved and your messages are sent much faster. SMTP Pipelining should always be used if supported by your email sending program and outbound email service (LuxSci High Volume supports SMTP Pipelining).
Multiple Recipients in One Message
Imaging sending the same message to 1000 recipients. If your send these one at a time and it takes 1 second to send each one, then that is almost 20 minutes to send. However, if you instead include all of these recipients in the BCC line of one single message, then it will take only about 1-2 seconds to get the message uploaded to the server (though it will still take the server some time to actually deliver it to those recipients).
Sending messages to multiple recipients using BCC allows you to upload your messages to the server very much faster than otherwise.
There are two downsides, however:
- The received message may appear more SPAM-like, since the recipient would not see his/her email address as the “To” recipient. Bccs are more SPAM-like than messages individually addressed (because it is so much easier and faster to send this way).
- A single message to 1000 recipients may take longer to be delivered to all of those recipients as the mail server will not generally parallelize delivery to the recipients but will instead process them sequentially. If the delivery time is not very time sensitive, then this point can be discounted.
LuxSci’s High Volume servers allow you to send to up to 1000 recipients in each message. Customers with dedicated servers and clusters can have this limit increased arbitrarily.
Smaller Messages are Better
A significant factor in determining how fast messages can be delivered is how long it takes to upload each one to the server. To see the difference, lets look at an example — sending a PDF to 1000 people in 1000 separate messages. This PDF is 1 Megabyte in size.
Case 1 – the PDF is attached to the message and it takes 10 seconds to upload the large message to the mail server. This results in it taking 10,000 seconds (almost 3 hours) to send these messages (unless you use concurrent connections or multiple recipients/message).
Case 2 – the PDF is placed on a web site and a link to it included in each message instead. The resulting email message is only 10 KB in size (100 times smaller) and it is able to be sent about 100 times faster. That’s less than 2 minutes without any other optimization.
It is best to remove images and other attachments from bulk messages to decrease the size. Images can be hosted on a web site and displayed in the message by merely linking to the image, rather than including the image content every time. Attachments that are not sensitive in nature can be similarly hosted on a web site and linked to. Keeping your email message size down will have a big impact on sending speed.
Clean Mailing Lists are Important
Obviously you should only be sending bulk messages to addresses that have opted into your mailings and/or with whom you have established business relationships. This, and all that is implied, are standard terms for the use of any bulk mailing service.
However, mailing lists get stale as people change addresses, domain names go defunct, etc. It is very important to cull invalid email addresses off of your lists. Why?
- Bad Domains. Sending email to an email address whose domain name is no longer valid (e.g. which has expired) can be very slow as it can take much more time to determine that the domain is bad than to determine that it is good and where to deliver the mail. The delay caused by bad domain names can really slow down your sending.
- Defunct Addresses. Sending email to now-invalid email addresses looks like spamming. Recipient servers like Yahoo!, AOL, McAfee, etc., are very sensitive to the number of messages that come through addressed to defunct email addresses. If they see a lot of these, they will either block your emails, or slow down the rate at which they process them. This will result in more delays and/or non-delivery to valid recipients.
- Waste of Time. Obviously, sending messages to invalid recipients also just wastes time and money.
You should take advantage of any and all tools available to track what recipient email addresses are failing and to actively remove them from your mailing lists. LuxSci provides many such features — including ones which can automatically email you digests of all failing email addresses and which let you create your own reports.
Insecure is Faster than Secure
OK, while encrypting your username and password and message content is always a good thing and always recommended, this encryption will always slow things down to some extent. It requires extra processing on the part of the server and your sending machine. It also requires somewhat more bandwidth to transmit all of the data.
So, if you are looking to “eek” out every bit of speed, we would recommend not using TLS or SSL when connecting to your bulk SMTP server. However:
- Be sure that the username and password being used to authenticate the message sending is NOT used for anything else. I.e. it is not your administrator user, the password is not one of your “standard” passwords, etc. You must assume that this username and password could be compromised.
- Do not grant this user any permission except for sending email. In LuxSci, you can restrict it from using the web interface and any other services.
- Change the password often. Change the sending user’s password often. I.e. weekly.
- Use tools to check that noone else is using this user to connect to your SMTP service. I.e. LuxSci provides alerts and reports about logins (successes and failures), which you can use to be sure that no one else is accessing this user account.
In the worst case scenario, if you have followed these guidelines, the username could be only used to allow someone to send email though your account until you next change your password or until you hit your sending limits.
Use an Appropriate Email Program
Many programs that are good for regular email usage are terrible for sending email in bulk. E.g. don’t try to use Outlook, Thunderbird, Mac Mail, and similar programs to send bulk mail if you are interested in sending speed or efficiency. Why? Such programs
- Generally do not support concurrent connections
- Might not support SMTP Pipelining
- Cannot handle large mailing lists (e.g. more than 100s of recipients) well
- Get bogged down and can be very slow when sending many messages
They are just not designed for it or optimized for it. Instead, use a program explicitly designed for bulk mailing. I.e.
- LuxSci Spotlight Mailer
- GroupMail
- Interspire Email Marketer
- Atomic Mail Sender
- SendBlaster
- Email Marketing Director
See our
videos on how to set these programs up with High Volume email.
Increase Capacity!
Finally, if you must send to many recipients in a short time, you may need to increase your outbound server’s sending capacity. At LuxSci there are tiers of capacity:
- Shared – Your account shares a single server with multiple other accounts. The capacity of the server is shared and your sending throughput (i.e. maximum concurrent connections, maximum recipients/month, etc.) is more restricted so that you play nice with other customers. Your outbound IP reputation is also shared with others.
- Dedicated – This is like “shared”, except that the sending server and IP is yours, and yours alone. You get all of the capacity to yourself and thus can attain a much higher throughput. Your IP reputation is also not subject to other customer’s actions.
- Cluster – When the capacity of a single powerful server is insufficient, a cluster is the perfect solution. It consists of 2 or more (really, any number) of outbound servers behind a load balancer. The more servers you put in your cluster, the higher your throughput can be. As a “side effect”, you also end up with multiple sending IP addresses for reputation management and with fail over so your sending can be resilient against server failure.
Which one should you get? That really depends on the number of recipients you want to send per month. Also, if you have requirements to send to large numbers of recipients in a very short time, you may need a dedicated or cluster solution even if the overall number of recipients in a month is not excessively large. Contact us for help in determining the best solution for you.
