, Washington Technology 2003-07-09
Despite wide use across government, intrusion detection systems have no standardmetrics to measure their performance, according to a new report by the National
Institute of Standards and Technology.
The report ^ÓAn Overview of Issues in Testing Intrusion Detection Systems^Ô
concluded that there are no comprehensive and scientifically rigorous
methodologies to test the effectiveness of intrusion detection systems, which
monitor and analyze systems and network traffic for possible hacker attackers or
Internet Security Systems Inc., Network Associates Inc. and Symantec Corp. are
among the vendors who sell intrusion detection systems.
NIST identified some probable metrics in the June report. They include:
Coverage: The range of attacks that a system could detect.
False alarms: The rate of false positives generated by a system.
Detection rate: The number of attacks a system can detect in a given period of
Resistance to attacks: The ability of the system to resist attacks to itself.
Throughput: How much traffic can the system handle at a given time.
Correlation: The ability to synthesize disparate events into a correct
recognition of attacks.
Detection of novel attacks: The ability of the system to detect attacks that have
not occurred before.
Detection of attack success: The ability to determine if the attack is
The report identified the work needed in each of these areas to develop metrics.