The open-source community is closer than ever to curing the spam problem, but they'll have to hold their noses and help out Windows users to get there.
Linux gained a toehold in corporate environments almost entirely on the strength of its interoperability, and that means solving the problems of clients running on other operating systems.
The situation is so bad that the engineers in the Internet Research Task Force have formed an
Many statistics, and my own personal experience, indicate that spam is nearly half of all e-mail traffic today -- and that number is only going up. Robert Banz, technical architect at the University of Maryland, Baltimore County (UMBC), tells me that last week spam made up 43% of the university's 885,000 e-mail messages.
Finding and tagging or blocking spam wastes a lot of system administrator time: in addition to denying all SMTP access to particular well-established offenders, sysadmins must also build and test new versions of anti-spam software, and integrate these programs into existing e-mail architectures. Once you add in the time spent researching new anti-spam approaches and packages, writing documentation, and supporting users, spam can take a huge toll on a sysadmin's productivity.
Users' time, however, is more important. For most people, e-mail is a tool used to accomplish other tasks, rather than an end in itself. If spam makes e-mail too much of a hassle, they'll find other ways to accomplish their goals. If people don't use our systems, we're out of work.
The most promising new technique for sorting out spam is mathematical: the software takes note of words used in spam and non-spam e-mail, and uses these results to guess if a particular message is junk mail. If the system guesses wrong, the user corrects it, and results are factored in to future calculations.
Apple's OS X mail client and the forthcoming version 1.3 of
But if your users read mail using a different client, like Microsoft Outlook, you won't find it easy to work with the statistical filters.
And that's a problem. While all of these statistical anti-spam tools are relatively easy to configure for site-wide operation, they need constant feedback in order to be even marginally effective. Every site can hack up some individual tools to provide feedback -- I've written several myself that allow feedback via special e-mail submission addresses or by saving mail to particular IMAP mailboxes. But tools like these tend to be site-specific, and are typically hacked together with Perl or shell scripts -- hardly suitable for a general release.
This scarcity of interoperable spam solutions is surprising and frustrating. Eric Raymond once claimed that the open-source community is good at scratching its itches -- that is, writing code to solve problems that directly affect us. This situation suggests that we might have a difficult time recognizing our itches.
When spam is half of our users' inboxes, that is our problem -- even if we, personally, have tools that let us manage it.
Moreover, Linux gained a toehold in corporate environments almost entirely on the strength of its interoperability, and that means solving the problems of clients running on other operating systems.
How can we best solve the interoperability problem with regard to spam solutions? Web interfaces for Unix anti-spam tools are the first step. Many organizations using the heuristic-based
But the real solution probably involves open-source plug-ins for commercial e-mail programs, most notably Outlook variants. Several companies have client-side ports of SpamAssassin, and several others even have ports of the technology to other server platforms. This suggests to me that the underlying technology is strong.
Why provide aid and comfort to users of proprietary, closed-source e-mail applications? For the same reason that we service these clients via SMTP, POP3, and HTTP: open source provides robust infrastructure via well-standardized protocols.
Hopefully the new IRTF Anti-Spam Working Group will look at standardizing an interface for anti-spam software. If open-source can't provide a simple, standard method for users to get rid of spam, end-users will find someone else to write software for their systems.