While NSS is a good organization, I have to wonder about a test that would rate TippingPoint so poorly and SourceFire and ISS so positively. My experience has been that while all are capable products, SourceFire and ISS take a LOT of effort to make them effective while TP, comparatively, takes much less effort. The article even mentions that fact, that Sourcefire took a lot of tuning. And having worked with all of these IPS, I have found that TP, SourceFire, ISS and McAfee are all pretty much the same in terms of effectiveness.

This is also odd, since a few years ago, NSS was giving TP glowing reviews. From 2004, NSS wrote:

"Overall the performance of UnityOne is very impressive, combining near-perfect security effectiveness with latency close to that of a layer 2 switch...we also found UnityOne to be very stable, surviving our extended reliability tests without missing a beat, and without blocking any legitimate traffic or succumbing to common evasion techniques."

That is from their report in January 2004. Okay, that was 5 years ago, times change.

One thing that always concerns me about these tests is the fact that they are laboratory-style tests and not real-world tests. Merely stopping an attack only one measure of effectiveness for an IPS. These devices must be made operational in a IT department. And they must integrate with other procedures, practices and devices. This is something a lab test cannot uncover.

Andrew Plato, CISSP, CISM, QSA
President/Principal Consultant
Anitian Enterprise Security

-----Original Message-----
From: listbounce (at) securityfocus (dot) com [email concealed] [mailto:listbounce (at) securityfocus (dot) com [email concealed]] On Behalf Of wickedpokah (at) gmail (dot) com [email concealed]
Sent: Wednesday, December 09, 2009 6:21 AM
To: focus-ids (at) securityfocus (dot) com [email concealed]
Subject: Re: Re: I love the smell of whining in the morning...

This is what TP is referring to:

http://www.networkworld.com/news/2009/120709-ips-tests.html?hpg1=bn

I have seen the full report and it is fair to say that TP did very poor compared to the competition. This is really no surprise for those of us close to the industry who understand TP's heritage and approach. exploit driven coverage with limited evasion capabilities wrapped around a pretty UI is a recipe for security by obscurity. Well, at least they beat out Juniper :)

Oh and NSS is probably the best and most neutral IPS testing body out there by far. This particular report is 100% independent and extremely comprehensive (the best I've seen to date) and includes coverage, performance, Evasion, and various TCO rankings. I *highly* recommend you obtain the report if you are interested and have the money to do so...

-----------------------------------------------------------------
Securing Your Online Data Transfer with SSL.
A guide to understanding SSL certificates, how they operate and their application. By making use of an SSL certificate on your web server, you can securely collect sensitive information online, and increase business by giving your customers confidence that their transactions are safe.
http://www.dinclinx.com/Redirect.aspx?36;5001;25;1371;0;1;946;9a80e04e1a
17f194

-----------------------------------------------------------------
Securing Your Online Data Transfer with SSL.
A guide to understanding SSL certificates, how they operate and their application. By making use of an SSL certificate on your web server, you can securely collect sensitive information online, and increase business by giving your customers confidence that their transactions are safe.
http://www.dinclinx.com/Redirect.aspx?36;5001;25;1371;0;1;946;9a80e04e1a
17f194

[ reply ]