Microsoft researchers released details on Thursday about the software giant's Search Defender project, a tool created to discover the major sources of comment spam on the Internet.
The research project aims to remove the advantages that spammers gain through more efficient comment spamming techniques, looking for large networks that feed to central domains and asking Web hosters and search engines to shut down the doorway sites to those domains.
"Comment spamming is getting worse and worse, causing a lot of clutter," Yi-Min Wang, manager of Microsoft's Cybersecurity and Systems Management Research Group, said in an interview with SecurityFocus. "In this particular report, we basically describe an automatic method to find (the spammers) and we need the communities to help actually fight them."
The Search Defender system first creates a list of potential doorway sites--intermediate destinations that take victims who click on comment spam to a more central Web page--with a tool called SpamHunter which crawls the Web for comment spam using search engine queries to find other sites in the same network. Then, using another Microsoft research project, Strider URL Tracer, the system finds the central domains to which the doorway sites refer visitors.
Among the results, the Search Defender system found that a large network of comment spam sent surfers to 17,000 Web pages registered on Google's Blogger, about 45 percent of which referred victims to six central spamming domains. Another case found that, among the more than 5,500 spam-related sites on Blog4Ever that were identified by SpamHunter, more than 97 percent used the same Google AdSense affiliate identifier, indicating the network was likely the work of a single comment spammer.
The project uses the efficiency techniques adopted by spammers against them, said Wang, who also worked on Microsoft's HoneyMonkey project. The more comments that link back to a spammer's site, the easier it is for SpamHunter to find it. Also, central domains that have a larger network of doorway sites are given a higher priority on the Search Defender results list.
Microsoft urged bloggers and forum moderators to be more vigilant about abuse of their systems, so as not to aid the spammers, and asked search engines to consider filtering out Web pages that host comment spam.
Posted by: Robert Lemos