ARC and SMTP MTA-STS: The State of Domain-based Email Authentication – Part 3

We’ll close (for now) our three part series on the state of domain-based authentication for emails by completing the story on technologies being deployed or defined to improve the security of the email ecosystem. In Part 1, we wrote about using Sender Policy Framework (SPF) and DomainKeys Identified Mail (DKIM) to authenticate the sending mail server. Part 2 described how Domain-based Message Authentication, Reporting and Conformance (DMARC) is used to provide clear guidelines for the treatment of mail that fail SPF and/or DKIM authentication.

In this post, we’ll touch on two topics that are mature works in progress in the IETF, the technical standardization organization that has brought us so much of the protocols that govern the internet. The first technology is Authenticated Received Chain (ARC), defined to handle the shortcomings of SPF and DKIM when used with mail forwarders or mailing lists. The second technology is about correcting the lack of security between Message Transfer Agents (MTA), and a solution to enforce strict transport layer security for SMTP message transfer between MTAs.

It’s worth reiterating again that all these technologies are building blocks, and only when used and deployed collectively by the entire ecosystem can we hope to create the barriers needed to thwart fake emails and mail surveillance by malicious actors.

Authenticated Received Chain

ARC, currently a standards-track IETF draft, tackles the issue that causes SPF and DKIM to fail when a mail receiver forwards the mail or the receiver is a mailing list. If the DMARC policy is set to p=reject, perfectly legitimate messages could fail validation at the ultimate receiver’s MTA and not be delivered. To briefly recap the scenarios:

• SPF verifies that the sending MTA’s IP address matches one of those contained in the DNS SPF TXT Resource Record (RR) for the domain identified in the MAIL FROM: When a mail is forwarded, the forwarding MTA’s domain as provided in the MAIL FROM: address will be used to validate against the allowed IP addresses that can send for that domain. The SPF authentication may very well pass. However, the message would fail the DMARC address alignment criterion as the MAIL FROM: domain would not match the domain in the From: address.
• A message whose body and some headers such as Subject: are cryptographically signed by a DKIM private key will fail signature verification at the MTAs for subscribers of a mailing list. That’s because mail lists typically add footers to the message body (e.g., how to unsubscribe from the list) and add the list name to the subject line. The calculated DKIM signatures of the changed message fields will not match those in the DKIM header.

The ARC processor at the receiving MTA takes a snapshot of key portions of a message before forwarding it or handing it off to a mailing list. It records the results of SPF, DKIM and DMARC validation, which it adds as a new ARC-Authentication-Results header to the message. It also signs the message body and adds the signature to an ARC-Message-Signature header. Finally, it signs selected header fields in the message such as the To:, From:, Subject:, plus the other two ARC headers and places these in an ARC-Seal header[1]. The public key used to verify these signatures at a receiver can be obtained from a DNS TXT RR just like that for DKIM. As was the case with DKIM, these signatures “prove” the identity of the forwarder or list server.

Such a message received at the ultimate receiver’s MTA will most certainly fail SPF and DKIM validation, and therefore may be subject to a DMARC p=reject policy (if that’s how it is set). However, even in this case the receiver can check the signatures in the ARC headers and verify if the
ARC-Authentication-Results parameter with the SPF/DKIM/DMARC results have remained unchanged since insertion by the forwarder or list server. If unchanged, it could use this as a signal to accept the message despite it not meeting the other validation criteria. In other words, the ultimate receiver is essentially using ARC validation to trust the forwarder or mailing list server as having properly vetted the original received mail.

ARC is not a perfect solution, as malicious intermediaries might change or drop ARC headers. However, by identifying changes made by intermediaries and providing a way to validate these, ARC offers a way to override a blanket DMARC mail rejection policy.  Thus, with the deployment of ARC, more email providers can set a strict(er) DMARC policy knowing that ARC offers them a way to avoid situations where legitimate forwarded mail might mistakenly be rejected.

Google and AOL have started to validate messages they receive with ARC headers and Gmail adds ARC headers to forwarded mail. There are several ARC implementations and a test suite; so there’s enough out there to get familiar with ARC and prepare for its deployment while the specification moves towards finalization.

SMTP MTA Strict Transport Security

SMTP MTA Strict Transport Security (MTA-STS) attempts for email what HTTPS Everywhere does for the web – make the use of SSL/TLS the default at first, with the goal of making it mandatory. Currently, it is commonplace for SSL/TLS to be turned on for the interactions between an email sender and its MTA, as well as an email receiver and its POP3/IMAP server. However, the path between MTAs typically remains unencrypted. This leaves email vulnerable to man-in-the-middle (MITM) attacks as well as eavesdropping.

There are several reasons for this:

• The SMTP specification does not require mandatory support for SSL/TLS, to allow for interoperability with legacy MTAs;
• A receiver’s support of the SMTP STARTTLS option can be trivially removed by a MITM[2], as the message is in clear text;
• If any aspect of the STARTTLS command is undecipherable, encryption is not used. It is trivially easy for a MITM to inject some garbage into the STARTTLS handshake (done in clear text) and have the connection downgrade to clear text;
• Even when the SMTP STARTTLS option is offered and accepted, the certificate presented during the TLS handshake is not checked to see if it for the expected domain, and unexpired. Most MTAs offer self-signed certificates as a pro forma Thus, at best we have an encrypted channel to an unauthenticated MTA, which can only prevent passive eavesdropping;

These show that the main issue with enforcing TLS on the inter-MTA path is that the receiving MTA cannot reliably communicate its security policy on the use of TLS, while the sending side has to guess if such a policy is in place.

SMTP MTA-STS provides a way for the receiving MTA to define its policy and make it available to all connecting MTAs:

• The MTA publishes a DNS TXT resource record under a specially defined sub-domain, _mts-sts, of the receiver’s domain containing an id, which is an alphanumeric value identifying a version number for the policy.
• The receiving domain hosts a policy server accessible over HTTPS containing its SMTP security policy. The policy includes the following fields:
  • A mode parameter indicating the receiving MTA’s policy for TLS on connections to it;
  • A max age parameter, showing how long the policy is valid for;
  • mx parameters, showing the DNS MX record names that are secured by this policy.

A sending MTA will query the DNS as a part of its normal processing to find the address of the receiving MTA. An STS-aware MTA will also obtain the TXT record and cache the STS policy version received. In subsequent interactions, it checks this policy cache to see if there is an existing, unexpired policy for the receiving domain; otherwise it can query the DNS record to see if there is a new version of the policy.

On first contact with a particular MTA, the sending MTA will contact the MTA TLS policy server for the given domain (example.com, say) at a specially defined HTTPS address to obtain the MTA’s TLS policy: https://mta-sts.example.com/.well-known/mta-json.txt .  If the policy’s mode field is set to “enforce”, the sending MTA must drop the connection and not deliver the mail if the subsequent STARTTLS handshake does not complete successfully. The domain name(s) received in the mx field must match that provided in the TLS certificate. This means that a proper, unexpired X.509 certificate must be presented and validated per normal certificate validation rules.

There is another solution of inter-MTA security, which goes by the mouthful SMTP Security via Opportunistic DNS-based Authentication of Named Entities (DANE) Transport Layer Security, simplified to MTA-DANE. It is actually a full-fledged IETF standard, RFC 7672. Instead of using a separate policy server, this solution provides the policy directly in a DNSSEC-protected DNS TLSA record. We won’t elaborate further on MTA-DANE because it is not used by most major mail providers (Google, Yahoo, Microsoft, Comcast, etc.) owing to its mandatory use of DNSSEC, a solution that has not gained much traction.

Google provides interesting statistics on the percentage of inbound and outbound mails from the gmail.com domain that are encrypted in transit. Since the start of this year, both are at ~ 88%. As far as regions go, 100% of the mails to/from Google from/to the top US based domains (e.g., aol.com, comcast.net, etc.) are protected. A part of this can be Google shaming providers by alerting Gmail users with icons suggesting that an email can be sniffed in transit. The deployment is also at 100% for top EU domains, while the situation is more mixed in Asia and Oceania.

Summing up the series

Absolute security is simply not possible with standards-based email solutions. There are proprietary solutions that claim complete end-to-end security. However, as with anything that’s not based on standard technologies, and users are stuck with vendor-specific solutions and platforms.

Standardized email security technologies are like a toolbox. These must be used consistently and widely deployed to be effective. A progressively secure email solution builds on a series of steps:

• PGP and S/MIME can be used to encrypt message bodies end-to-end, but the certificate management needed can deter most users;
• However, even if the message body is protected, the “metadata” of an email message (the sender, the receiver, the subject, etc.) remains in the clear. The use of MTA-STS as described here can mitigate surveillance of these (as well as any unencrypted message bodies) while the email is in transit over the internet;
• SPF and DKIM, as described in our earlier post, can authenticate sending MTAs and cut down on spam and phishing from rogue MTAs;
• DMARC, described in another previous post, allows sending MTAs to more firmly guide receiver MTAs on how suspicious messages that fail SPF/DKIM/DMARC validation should be handled;
• ARC, described above, allows receivers of forwarded mails or subscribers of mailing lists, to accept mail from legitimate sources even if such mail fail SPF/DKIM/DMARC validations.

No doubt the future will provide additional tools to add to our repertory of email security solutions – perhaps even a standard solution for perfect end-to-end security! But don’t hold your breath – that may take some time.

Do you have questions about email or email security?

Ask Erik

Notes

[1] The “chain” in the ARC name refers to the fact that there might be multiple intermediate MTAs, each of whom can add ARC headers. Each intermediary and the ultimate receiving MTA can validate the chain of such “seals” before deciding on further message handling. The reader should consult the specification for details on this.

[2] It should be mentioned that removing the STARTTLS option is not just the actions of a malicious MITM. Firewalls and virus scanners also do so, as a feature!