DMARC: The State of Domain-based Email Authentication – Part 2

Building a safer email ecosystem with DMARC

In our previous post, we described two techniques for authenticating an email sender:

• Sender Policy Framework (SPF), IETF RFC 7208, which verifies if the sending MTA is indeed authorized to send mail on behalf of a domain; and
• DomainKeys Identified Mail (DKIM), IETF RFC 6376, where a domain shows “ownership” of a mail it sends by signing portions of it so that critical aspects cannot be forged by intermediaries.

Like most technologies, these are just individual weapons in the arsenal for fighting phishing and spam. Weapons, like all tools, need to be properly used if they are to be effective. Unfortunately, as we described in the earlier post, both SPF and DKIM are deployed in a manner that reduces their usefulness. With SPF, the validation policy set by the sender is often chosen in a manner that leaves handling authentication failures at the discretion of the recipient. DKIM, on the other hand, does not even have an explicit policy directive set by the sender. Moreover, in a heterogeneous mail environment, some perfectly legitimate MTAs might not be capable of signing messages.

Thus, receivers in actual deployments tend to “soft fail” any SPF and/or DKIM validation failures as there are reasonable situations when legitimate mail can fail such checks. A common example is forwarded mail (which fails SPF), or mail sent via a mailing list (which fails DKIM). Mail providers consider it better to deliver most mail (even if some are fake or spammy) rather than risk dropping legitimate mail. Thus, neither of these techniques individually or combined provide clear guidance to receivers, and the resulting actions can be inconsistent.

Also, neither SPF nor DKIM include an explicit indication by which a receiver knows that the sender uses these protocols and expects the receiver to handle mail according to those policies.  A receiver could, as a general rule, access the sending domain’s DNS TXT Resource Record (RR) and poke around in search of SPF or DKIM related settings for every received email. However, doing this unprompted is at the moment very unusual, ad hoc behavior.

Moreover, even if present, senders do not know how receivers handle their setting of SPF or DKIM policies. Are messages being received from MTAs the sender is unaware of?  It would seem reasonable to ask for a report to be sent back from the receiver indicating how it treats problematic message, and why.

A new protocol, Domain-based Message Authentication, Reporting and Conformance (DMARC), IETF RFC 7489, has been defined to create an email ecosystem where senders and receivers collectively agree to conform to certain clear guidelines for handling spam and fake emails. Conforming senders set a SPF record and/or a DKIM signature against which incoming emails should be validated by conforming receivers, with unambiguous semantics for failure conditions. Furthermore, receivers report back to senders about instances of validation failures. Prior to DMARC, senders and receivers were decoupled and each operated independently. With support of DMARC by senders and receivers, there is the expectation that email authentication will be handled consistently by all conforming entities. We devote the rest of this post to explaining DMARC and providing information on its deployment.

Domain-based Message Authentication, Reporting and Conformance (DMARC)

Domain-based Message Authentication, Reporting and Conformance (DMARC), goes directly to the heart of the fake email issue – the spoofing of the From: address (specifically, that defined in IETF RFC 5322) by a rogue sender. End users cannot be expected to validate that a message is truly sent by the claimed entity, especially if all visible aspects of the message conform to expectations. Both SPF and DKIM, however, tie their validation to other identifiers for the sending domain. These identifiers are deep within the message envelope, something that most email users rarely look at, and, indeed, should not be expected to.

SPF validates that a message was sent by an MTA with an IP address that is one of those identified in the DNS TXT Resource Record (RR) for the sending domain presented in the initial handshake exchange. This is the domain provided in the MAIL FROM: field defined in IETF RFC 5321 and carried in the message envelope. However, it is easy for an imposter to create a rogue domain and publish a DNS SPF RR of its MTA’s IP address. The SPF validation would pass. DKIM can also be subverted in a similar manner. If present, a rogue domain can sign various headers which signatures that can be validated against its public key provided in its DNS DKIM RR.

With DMARC, authentication occurs at two levels – first, messages are checked for SPF and DKIM violations. These results are then input to the critical step introduced by DMARC – the alignment of the various identities used by these protocols.  The DMARC validation requires that the domain in the visible From: field matches the domain used in the MAIL FROM: field used for accessing the SPF record. A similar check is done for DKIM. The message and/or headers must be signed with a valid signature using the public key provided in the sender’s DKIM record. Moreover, the domain in the d= field of the DKIM-Signature header must be the same as that in the From: field. The following table shows the various possibilities that can determine if the DMARC validation check passes or fails.

SPF authentication OK?	Y	Y	N	Y	Y	N	N	N
SPF address alignment OK?	Y	Y	N	N	N	Y	Y	N
DKIM authentication OK?	Y	N	Y	Y	N	Y	N	N
DKIM address alignment OK?	Y	N	Y	N	Y	N	Y	N
DMARC validation	Pass	Pass	Pass	Fail	Fail	Fail	Fail	Fail

 

Note that failing both aspects (i.e., the protocol-specific policy and the alignment of identifiers) of either SPF or DKIM authentication does not mean that DMARC validation fails. That’s because of the legitimate reasons we pointed out earlier why these authentication checks might fail. If one or the other protocol fully passes both aspects, this is sufficient to pass DMARC. However, failure of any aspect of both SPF and DKIM – either failing address alignment or the protocol-specific policy – is sufficient to fail DMARC authentication.

If DMARC validation fails, the mail is handled according to the failure policy defined in the sender’s DNS DMARC RR. An entry here defines how the sender would like a received mail that fails DMARC validation to be handled. It is no longer up to the receiver’s discretion. The options available to a receiver are “none”, “quarantine” and “reject”. The first option means that no policy has been defined, and is typically used when first setting up DMARC to allow a sender to gain information on the types of messages being sent from its domain and how these are handled. The “quarantine” option asks the receiver to treat the email as suspicious, which usually means putting it in a spam folder. “Reject” requires that a mail failing DMARC validation must not be delivered.

In all cases, the DNS DMARC RR identifies return addresses to which the receiver must send reports of all failures – either aggregated or individually. Senders can review these reports to find out what messages are not authenticating, and take appropriate actions to maximize delivery.

The following figure provides an overview of the players in the DMARC-based email ecosystem and how these interact.

In step 1, a sending domain owner populates its DNS with TXT RRs for SPF, DKIM and DMARC. It also needs to ensure that MTAs have the appropriate private key for creating DKIM signatures. An email sent from that domain (step 2) has a DKIM header appended by the sending MTA (step 3). Upon receipt at the receiving MTA, the sending domain’s DNS is accessed to obtain its SPF, DKIM and DMARC policies (step 4). After the policies are evaluated (step 5), a successful validation leads to the message being delivered (step 6). Failures are reported to the sending domain (step 7).

In summary, DMARC defines a collaborative framework where sending and receiving domains do not act independently or inconsistently. A feedback loop ensures that senders are apprised of the receivers’ actions, allowing fine tuning of sender policies so as to increase the delivery rate of valid emails while thwarting phishing and spam mails.

DMARC Deployment Status

So as to prod receivers into joining this DMARC-based ecosystem, large email providers like Yahoo and Gmail announced that they will reject messages from domains that do not set a DNS DMARC RR. Also, Gmail is expected to change its DMARC policy from “none” to “reject” sometime in 2017. This means that users with a gmail.com address in the From: line will find their messages rejected unless these messages are sent via Gmail. Yahoo has had a similar policy in place since 2016.

These changes can have implications for small medical practices that might use a Gmail or Yahoo address to send medical newsletters using a small email provider. If receivers of these newsletters have a DMARC policy in place, such emails will fail the DMARC address alignment test and be rejected.

At the moment, though, many organizations (including very large ones) have adopted DMARC in the monitoring mode (setting the authentication failure policy to “none”). DMARC.org identifies those who have moved from “none” to “quarantine”, while others have taken the final step of moving from “quarantine” to “reject”.

A report by ReturnPath offers some interesting data from 2016: Of 1000 global brands from 33 countries surveyed, 29% provide a DMARC record, with US brands leading with an adoption rate of 42%. 70% of global consumer mailboxes (~2.5 billion) are protected by DMARC. That’s 80% of US consumer mailboxes. Unfortunately, when viewed by industry sectors, healthcare has the lowest adoption – and has been for two years in a row. Of those in this sector that do set a DMARC record, 83% set their policy to “none” which effectively does nothing.

Unfortunately, most US government organizations also do not have proper DMARC records, although it is heartening to see that the Social Security Administration (ssa.gov) and the FBI do!

In our next post, we’ll conclude this series with a description of technologies in the process of being defined. One is ARC, Authenticated Received Chain, which is a companion specification to DKIM and DMARC being defined in the IETF which solves the authentication problem with email lists. Another is to ensure full SSL/TLS encryption between MTAs, thereby ensuring the email equivalent to HTTPS Anywhere for the Web.