Hi Dave,

It's always good to hear from you.

With regards to your comments:

In your reply, you wrote: "APWG has not revised our methodology for counting."
This seems to be contradicted by the APWG_Phishing_Activity_Report-Oct2004.pdf which begins by saying:
"With this report for October 2004, we are introducing a new methodology which provides a measurement of phishing activity based on the number of fraudulent "baiting" websites extracted from phishing email messages, in lieu of counting the email messages themselves as presented in previous reports."

You also mentioned, "APWG contribution of phishing attacks is not linearly correlated to the number of members, although there is no doubt some correllation[sic]." The point that I was trying to make in this article was that there is a bias, but the bias is not disclosed in any of the APWG reports.

You also wrote, "The NANAS numbers are clearly nonsense. Recording 17 phishing sites in all of December 2006 is clear evidence that this is a useless metric." Perhaps you misread page 3. There were 17 phishing emails on 29-Dec-2006. That is one day, not the entire month. As I also wrote (on page 1), NANAS contains a sampling of spam submitted by specific groups. It is therefore inherently biased. However, increases in any particular type of spam (e.g., pump-and-dump, porn, or phishing) result in an increase in the samples seen on NANAS. The purpose of NANAS is not to shutdown spammers. The purpose is to provide a sample of current spam messages (and other online abuses). You can read the full charter for NANAS at http://www.killfile.org/~tskirvin/faqs/nanas.html -- nowhere do they talk about using NANAS to stop spam. Perhaps you misunderstood this when you wrote, "The NANAS list doesn't help stop phishing, so people stopped contributing." As an aside, the volume of postings on NANAS has continued to increase since the forum began in 1996, so people are definitely still contributing.

I certainly do not doubt that "APWG members received more than 500 times that number of unique phishing sites and almost 2000 times that number of unique phishing emails." (In fact, I suspect that this is an underestimate.) The question that I asked related to the percentage of total and number of mailing campaigns. For example, if you suddenly see a mailing campaign that generates 100,000 phishing emails with a variety of servers and subject lines, how do you know that they are not part of the same mass mailing? Can you tell if they are general-purpose (blast-o-gram) or targeted (spear-phishing)? Similarly, can you estimate the number of phishing emails that you are missing? These would be the ones that get past the spam filters or that take routes that are not part of your detection network. Keep in mind, I never said that phishing had gone away. I only stated that blast-o-gram phishing has significantly dropped in volume.

You mentioned many different sources for phishing information. As you pointed out, Postini and MessageLabs have regional biases. Unfortunately, their sites do not say how they get their data nor do they declare their collection biases or analysis methods. Perhaps what we need is something similar to the Internet Storm Center's World Map (provided by Dshield). This shows attack volume by region. It would also be worthwhile for different companies to share information and compare results. For example, CastleCops could share their raw data with Postini and MessageLabs in order to see if they generate the same numbers when given the same data. If they cannot generate the same numbers, then the numbers cannot be directly compared.

Neal Krawetz, Ph.D.
Hacker Factor Solutions
http://www.hackerfactor.com/
Author of "Introduction to Network Security" (Charles River Media, 2006)
and "Hacking Ubuntu" (Wiley, 2007)

[ reply ]

Link to this comment: http://www.securityfocus.com/comments/columns/435/34384#34384