I think it's right to question what these 14 evaluations did, and why they missed these problems. I think it's wrong to assume these evaluations were useless and the people who did them aren't providing value.

The reality is, Litchfield is an expert who had incentives to try and find problems: he is writing a tool that's supposed to evaluate (scan) Oracle. If you run Nessus against your site and it shows no problems, does that mean you're completely safe? No. If you have other problems, which assuredly you do, does that also imply Nessus (or Nsat or Ncat or eEye or whatever) is useless? No.

The more realistic problem is that these 14 evaluations were all too similar. So running more of the same is likely to generate results that are more of the same. If you're really trying to find problems, then you need to increase the probability that new tools (evaluations) are looking for different things so the space of things you're considering and testing is more/wider.

[ reply ]

Link to this comment: http://www.securityfocus.com/comments/articles/309/9947#9947