Interview with Nate Fitzgerald, Product Manager, Premium Email Archival
LuxSci recently had the opportunity to speak with Nate Fitzgerald, Product Manager of Message Archiving at MX Logic. Nate has been with MX Logic for over 4 years and is responsible for their successful archival service. His insight has been invaluable in helping LuxSci get up to speed with all of the technical nuances of this service. We would like to share with you his knowledge and thoughts regarding the security, reliability, and utility of LuxSci’s Premium Email Archival service.
LuxSci is well-known for its premium and secure email services. Two important components of a premium email service are email archival and email filtering. While LuxSci has its own in-house solutions – “basic” filtering and “basic” archival – we long ago realized that both services are very specialized. Partnering with a company whose sole focus is premium-grade filtering and archival is a win-win for LuxSci and its customers. LuxSci’s mission is to offer the best available products to its customers that focus on security, technical support, and breadth of services.
In 2003, LuxSci partnered with MX Logic to offer its customers Premium Email Filtering. As one of their first Resellers, LuxSci has in-depth knowledge of their filtering and archival products. Today, MX Logic is recognized as one of the best providers of email filtering in the industry, for both quality of service and robustness of feature set. In the last couple of years, they also added Premium Email Archival that LuxSci has integrated into its own product line.
The benefits of LuxSci’s longterm partnership with MX Logic are:
- Extensive experience supporting MX Logic’s products (comparable to MX Logic’s senior support staff). We provide our customers with a real value-add in terms of support and timeliness of response.
- Extensively integrated Premium Email Filtering and Archival offerings into our own products that translate into a seamless turn-key offering at LuxSci.
- Offering MX Logic services at a significant discount over MSRP due to the volume of licenses we sell.
The Interview
The email archival solution stores archived email messages in two redundant data centers, both in Colorado and within driving distance of each other.
Can you discuss the location of these data centers and how they were chosen?
Many of our services, including Message Archiving, require real time communications between the redundant data centers. This is mainly due to the fact that we run both data centers simultaneously which gives us faster failover in the event of a disaster. There is a significant cost benefit to having them located far enough from one another that regional issues aren’t a factor, but close enough that fast, inexpensive metro data lines are within reach.
Are there plans for redundant storage in another state or country?
Not at this time. Average latencies in the US are very small even from one coast to another and our analysis and those of the 3rd parties that audit us have determined that the current architecture cannot be significantly improved by the addition of another data center in terms of both redundancy and performance.
What dependencies are there between these data centers? What resources do they share? I.e. what, if anything, affects one data center that could also affect the other?
Nothing, they aren’t even owned by the same companies.
When you do software updates, how do you roll these out to minimize issues?
Under normal circumstances, software is released in scheduled releases approximately every two months. We also have the ability to release unscheduled patches in the event that it makes sense to release something outside of the normal cycle to minimize the impact of operational issues to our customers.
In either case, before we release anything, the code is tested by our Quality Assurance department. Several departments are then shown a feature demonstration of new functionality to insure that all of the appropriate advance communications are sent out to our customers and partners in a timely manner. All maintenance windows and customer-impacting software changes are announced via our formal communications channel to each partner.
In terms of the storage of the email messages in the archives…
How are the messages actually stored? I.e. in what format? Are they encrypted?
Messages are stored exactly as we received them from the customers’ service. They are never altered and we preserve those originals in case the index ever needs to be rebuilt. All messages are encrypted when stored using 256-bit file-based encryption; SSL or TLS encryption is used for message content transmission, assuming the customer’s system supports it. Even the search index itself is encrypted.
What kinds of physical and software security protect the archived data as it is stored?
Both data centers employ impressive physical security measures preventing unauthorized personnel into our cages. Obviously, one customer can’t see another customer’s messages via our secure portal. Even end users are restricted to seeing only their own messages.
Who exactly has access to the raw data?
Select operations personnel have access to the storage systems for two reasons:
- Routine maintenance: We have a duty to ensure that our systems are performing in a reliable and satisfactory manner.
- Customer support requests: Occasionally, operations level support is required to identify the causes of some of the more fringe cases we see.
Both these scenarios protect customer privacy while ensuring that we can provide a high level of customer service for customers facing unusual issues with their service. Customer data is only accessed when there is a support ticket that explains why the data should be accessed. Most levels of troubleshooting can be performed using automated tools which produce troubleshooting data without revealing message content.
Email archival comes with “unlimited storage” for messages…
What kind of storage architecture is used that permits this kind of scaling, while still providing timely access and searching of message data?
The search index is like a digital table of contents used by the search engine to locate information stored in Message Archiving. Without the index, the search engine would have to search every word in every message each time a search is conducted, which would be far too resource intensive and slow to be practical. The index maintains a much smaller version of message content by representing common words and phrases with binary tokens. The result is a fully searchable index that is a fraction of the size of the data that it represents.
HIPAA requires that all protected data be transmitted securely at all times. I know that all email that is drawn out of LuxSci’s servers into the archives is transferred over a secure IMAP or POP connection.
How about when the data is transmitted within the archival system?
Data is moved over a secure connection.
I have heard a great deal about the extensive work that has gone into making sure that every message is archived without failure. I.e. a message will never be removed from the source servers in the client’s email account at LuxSci until it has been verified to be archived successfully in both archival data centers.
Can you discuss how this works and what is involved in making it possible? What kinds of issues does this protect against?
Very early in the design of the import mechanism we felt it necessary to be 100% sure that we would never delete a message from the client’s journal mailbox (where messages are stored temporarily prior to being automatically imported into the archives) unless it was archived safely in our data centers. We have achieved that even though it made the design a lot more challenging. Based on competitive research, we know that there are others in our industry who have no such safety mechanisms. Those systems rely on little more than a “wing and a prayer” to get each message safely into their archives.
What was/is the biggest technical issue involved in providing such a redundant and reliable archival system?
Very similar to challenges in Premium Email Filtering: Being able to process and index all the non-RFC compliant mail* that is created out on the internet and doing it in a cost-effective manner.
*non-RFC compliant mail is mail whose content is not formatted as it should/must be, based on how email is supposed to work. This is a common problem with Spam and bulk messages.
In terms of compliance (i.e. with Sarbanes-Oxley, Gramm-Leach-Bliley, NASD 3010, FRCP, SEC, etc.), many companies and organizations are required to keep copies of their email for some number of years. The Premium Email Archival service certainly allows you to keep copies of all sent and received email messages for 1, 3, 5, and 7 years. However:
How does the archival system actually provide compliance? What are the key things that it does that result in compliance?
The term ‘compliance’ has really become over used, and that has eroded its usefulness. There are thousands of versions of the definition of “compliant,” depending upon what laws or regulations that the customer is subject to. To make matters worse, the customer’s interpretation is often not the same as ours, making the term even more vague. One law that we look to very frequently to help define compliance is the SEC 17a-4 legislation. It is clear, well-practiced and strict, so it makes for a great standard to try to adhere to when you are in the business of helping thousands of customers with varying compliance needs. So when my reps talk about compliance with a prospect, they refer to the features mentioned below. It’s then up to the customer to determine if these meet their needs based upon their specific compliance requirements.
We often have customers that want us to make them compliant. No matter what we do, we can’t do that. Why? Compliance is almost always more than just email archival. Yes, we take care of one part of compliance which is email archiving, but there’s usually more to it than that.
For example, with HIPAA, “encryption” is king; however, if a customer seeking to be HIPAA-compliant sets up Message Archiving to import mail from their server without enabling encryption for all of the connections, they may be using our service in a non-compliant manner.
Similarly, if a customer has a requirement that client data be completely destroyed within 1 year from the time that the client ceases to be a client, then the customer is responsible to choose archival that lasts no longer than 1 year.
Here are the features that I was referring to:
- Tamper-proof read-only storage
- Dual data center storage
- Automatic quality verification
- Dual commit message capture
- Auditable message serialization
- Searchable content, attachments, & metadata
- Powerful transport & storage encryption
What can a client of Premium Email Archival say to his/her lawyer or auditing agency when asked about compliance? Is there a document or certification of some sort that says that if they are using this system properly, then they are in compliance with “X, Y, and Z”?
Amazingly, there is not. Not even the SEC has a formal certification process. The best you can do is to get endorsements from organizations or perhaps a well-known law firm, but that doesn’t relieve the customer from having their own legal advisers put their stamp of approval on the system. After all, they will be the ones to defend their client in court, not me, and not LuxSci. So, these types of endorsements have little or no actual value other than a lineitem on a presentation. The best thing to do is to evaluate your legal needs and then make sure that the features built into the archive you choose match up to those needs.
What happens to email after the expiration period?
The messages are securely purged from the system, meaning that they are gone forever and cannot be recovered, even in a lab.
Is it deleted from the system yearly, monthly, continuously? How does the message purging actually happen?
Messages are purged within 1 month of expiration. The average message ages 15 days past its retention period before being deleted.
What do people do when they want to export their email before it is deleted, or if they wish to move to a different type of archival solution and take their email with them?
If the customer wants to export up to a few thousands messages from the system, they can do that at any time on their own via the web console export function. We can assist with larger exports via a professional services engagement.
How can existing historical messages be imported into the email archival system?
We will import historical data almost identically to real-time journaled data, using one of our mail source connectors specially designated to the task of processing historical data. This feature will be available in early March ’09.
Specificity, historical data will be able to be imported via secure POP or IMAP connections to an account on the customer’s server which holds the historical email data. This data will be imported in the background in a way that is robust and reliable, no matter how much data is waiting to be downloaded.
LuxSci’s Premium Email Archival and Premium Email Filtering are both provided through our partnership with MX Logic and are both accessible via the same web-based interface.
In addition to the great price savings that users have available if they purchase both services, can you comment on the value of having both archival and filtering, and having them in particular through MX Logic?
First, the price is fantastic. We have had great success with our filtering plus archival bundle discount. Second, there is real value in doing business with one vendor for both services, especially as archiving requires that each customer have an anti-spam/anti-virus service because it is an unlimited storage product. We look forward to increasing the benefits of using MX Logic for both Email Defense and Premium Archival in the future including some exciting integration features.
Archival and Filtering are also tightly integrated with LuxSci-hosted email systems. This includes automatic firewall lock downs of email flow, simple setup of capturing inbound and outbound email so that it flows automatically and securely into the archives. Getting MX Logic archival and filtering with LuxSci email hosting provides a very robust combination.
How does MX Logic’s archival solution compare to that of some of your competition, such as ContentFast and Google (formerly Postini)?
We [MX Logic] are still partnered with ContentFast (for providing Archival to some customers, though the archival services discussed in this interview are completely unrelated to those of ContentFast) because their service supports some platforms and features that we don’t, including GroupWise and Domino. On the other hand, we feel that we have the most intuitive, easy to use service in the industry. We are also the only archive in the cloud that offers month-to-month contracts. We feel that this is a huge advantage to our customers who want to avoid large expenditures in these uncertain times.
I would also like to mention that our UI (user interface) is years ahead of the next closest competitor. Our customers and industry analysts have made it clear that we have a solid competitive advantage in this area.
Google still suffers from a very unreliable SMTP-based import scheme that virtually ensures that some messages will be untraceably lost in transit to its archives. They also require customers to maintain 30 days of mail cached on their servers. This sort of storage overhead is exactly what most of our customers are trying to avoid.
Can you give us any hints as to some of the things on the drawing board for email archival going forward?
We are always looking to expand existing features to run faster, handle more mail, larger mail, etc. so many of the things we add early this year will be focused on that. Later in the year, we will be focused on adding completely new features that should help customers manage their growing archives more efficiently.
Thank you very much, Nate!
Email updates:
