9 Guardian Digital | SPF Ready for Prime Time?


List of Tables
3-1. SRS Support in the Major MTA's

Chapter 1. Introduction

As of the time of this writing in the fight against SPAM a policy has been drafted to target sender address forging called SPF (Sender Policy Framework). The basic premise is to verify that the sender of an email is in fact who they by claim to be. If they are not then mail can be rejected. This could potentially eliminate a big percentage of SPAM and who wouldn't want that.. But there have been problems with SPF and it isn't the big solution that everyone had imagined when it first hit the scene. There are a couple of plaguing issues that keep it from becoming a mature solution with a standard.

Chapter 2. What is SPF?

The first version of SPF (also know as "Classic" SPF) was a creation of Meng Wong, founder of Pobox.com. In short the scheme is based on domains publishing what servers are allowed to send mail for themselves using DNS TXT records. A receiving MTA can then look at the domain the sender is claiming to be from and the IP address of the connecting client and check the SPF (DNS TXT) record for that domain and verify if the client is allowed to send mail for the said domain. From the results the receiving MTA can take appropriate actions. The goal is to prevent sender forgery, one of the most common characteristics of spam. SPF was a proposal considered by IETF's MARID group.

Chapter 3. What are the issues with SPF?

3.1. SPF Breaks Forwarding - SRS is the Solution?

SPF breaks email forwarding. If the original sender joe@domain1.com sends an email to a bill@domain2.com who is forwarding email to bill@domain3.com and the MTA at domain3.com is performing SPF checks the check will fail. This is because the original sending domain's (domain1.com) SPF records don't list the connecting client's (mail server of domain2.com) IP address as being authorized to send mail for domain1.com. For a diagram refer to http://spf.pobox.com/mailflows.html.

The proposed at spf.pobox.com are to use SRS (Sender Rewriting Scheme), but there are problems with SRS. One of which is that the current state of the SRS patches as reported at http://www.libsrs2.org/status.html is as follows:

Table 3-1. SRS Support in the Major MTA's

Courier Mail Server Finished
Postfix Broken
exim Internet Mailer In Testing
Sendmail Consortium In Testing
Microsoft Exchange No Data
Qmail Non-Existent

So SRS isn't quite ready and the spf.pobox.com's FAQ (http://spf.pobox.com/faq.html#forwarding) suggests that "Until the SRS patches are ready the following workarounds will preserve the important functionality". I have to disagree. To me the important functionality in regards to bounce handling is that the original sending account receives the bounce. The site suggests using procmail recipes in place of a .forward file. The "advanced" recipe will handle a bounce generated by the destination mail server by storing it in the forwarding address's mailbox and does not attempt to deliver the bounce to the original sender. To me that's not "preserving the important functionality". So I don't feel that there is an interim solution and since SRS isn't ready to use there currently isn't a solution in my mind.

3.2. Spammers are Using SPF

At this point in time it's no big secret that spammers are publishing their own SPF records to thwart the system and once their domains end up on a URI block list they throw them away and start with new domains with new SPF records. There may be future solutions to this such as "reputation" schemes (i.e. Aspen Framework) to judge a domain credibility but not at this time. (See Ch5. Sec5 below).

Chapter 4. Why isn't there a standard for SPF?

The MARID group (MTA Authorization Records in DNS) was created by the IETF (The Internet Engineering Task Force) to "produce a standard in the area of DNS-stored policies related to and accessible by MTAs." Due to a failure to come to an agreement to a solution the MARID group was dissolved as reported in an email from the IESG secretary (http://www.imc.org/ietf-mxcomp/mail-archive/msg05061.html). "From the outset, however, the working group participants have had fundamental disagreements on the nature of the record to be provided and the mechanism by which it would be checked. Technical discussion of the merits of these mechanisms has not swayed their proponents, and what data is available on existing deployments has not made one choice obviously superior. Each represents trade-offs, and the working group has not succeeded in establishing which trade-offs are the most appropriate for this purpose. These assessments have been difficult in part because they have been moved out of the realm of pure engineering by the need to evaluate IPR and licensing related to at least one proposal in the light of a variety of licenses associated with the deployed base of MTAs."

It seems that the problem came down to the "last call" when a proposed solution (Sender ID) to MARID included PRA (Purported Responsible Address algorithm) which Microsoft claimed intellectual property rights to. Microsoft was willing to allow free use but only in conjunction with a patent license. Most of the MARID participants objected to this and rightfully so. Many strongly suspected the intention to gain control over another piece of the industry.

Chapter 5. The Future

The future of email sender verification has several possibilities. Some of which are Yahoo's "DomainKeys", Cisco's "IIM" (Identified Internet Mail), a mix of both of these referred to as "DKIM", the "Aspen Framework" (which incorporates second generation "Unified" SPF) and CSV (Client SMTP Verification). Here is a brief run down on these proposals.

5.1. DomainKeys

DomainKeys uses public/private keys generated on a per domain basis. The sending mail server will sign the email message with the private key and append a header with the digital signature. The public key is published in DNS. The receiving server then accesses the public key and validates the signature. Once validated a reputation profile can be created for the sending domain and subsequently be incorporated into accreditation services.

5.2. Identified Internet Mail (IIM)

Identified Internet Mail is similar to DomainKeys one difference being that the public key is included with the digital signature in the appended header instead of storing it in DNS. Once the receiving MTA has verified a message's origin, the origin can be evaluated by third-party reputation and accreditation services.

5.3. Aspen Framework

Another Internet Draft authored by Meng Wong (SPF creator) has been submitted(http://xml.coverpages.org/draft-ietf-marid-rationale-00.txt) that includes the "Aspen Framework" which has the three components of Authentication (which incorporates "Unified SPF"), Reputation, and Accreditation. "Unified SPF" will "require senders and receivers to publish for and check the host name, mail-from, and Purported Responsible Address, a technique that checks the record against the most recent sender of the e-mail address."(http://news.zdnet.com/2100-1009_22-5380029.html).

5.4. Client SMTP Verification

According to http://www.bbiw.net/CSV/draft-ietf-marid-csv-csa-01.txt Client SMTP Verification performs two checks. The first is to verify that a client is authorized to send mail for the sending address and the second is to perform accreditation service checks for a "good" email rating.

5.5. Reputation/Accreditation Services

The above all reference the use of reputation/accreditation schemes which implies that these will be components of future email validation. In March 2005 the draft of "Server Index Query" protocol seems to support this(http://www.ietf.org/internet-drafts/draft-irtf-asrg-iar-howe-siq-01.txt). The intention of SIQ is to "provide a standard means by which a mail exchange (MX) server can query one or more external services for scoring based on facts or reputation of an IP/domain pair."

Chapter 6. Summary

I, as everyone else, would love to be able to block all SPAM and I certainly applaud all of the efforts that have been and are still being made. But it seems obvious that SPF alone isn't going to be the answer. It doesn't handle the forwarding issue and SRS isn't ready as a solution. One could argue that SPF can at least be used not to reject mail but to whitelist mail from senders that pass SPF checks. In view of spammers deploying SPF themselves this would actually be counter productive as it gives them a form of credibility.

Based on the material presented here there are options other than standalone SPF that on the surface seem to provide a better solution but the cost is that they are more complex in that they require reputation/accreditation services. But does the lack of agreement on the simpler SPF (which turned out to be not so simple once the forwarding issues surfaced) foreshadow the difficulties in standardizing more elaborate proposals? If the trend towards reputation/accreditation gains momentum, which by the way would still require some form of sender validation to be established (you can't build a dependable reputation of a sender when it can't be verified), harmony on the architecture of such services seems a very long way off. Sender verification is a problem that certainly needs to be addressed but SMTP wasn't originally designed with this functionality in mind. Therefore a viable solution is not going to be as simple as publishing DNS records of authorized mail servers. SPF on it's own isn't the answer.

Appendix A. Resources

Below are some resources that were referenced:

[ Company ] - [ Press ] - [ Contact ] - [ Partners ] - [ Store ] - [ Newsletters ] - [ Site Map ]
Copyright (c) 2000 - 2009 Guardian Digital, Inc. Linux Lockbox and EnGarde are Trademarks of Guardian Digital, Inc.