> First, I find it troubling that the history and full meaning
> of the term "target-based IDS" (which I coined in 2000) was
> omitted. That this article didn't review any fully
> target-based IDS products will almost certainly leave readers
> with a misunderstanding of what target-based IDS really is.
Cisco Threat Response (CTR) was designed to be an automated forensic
investigation and response tool. An artifact of this process is that it
eliminates false alarms, but this has never been the exclusive thrust of
the product. So in this context CTR was never designed to be a
target-based IDS or even a facilitator of that concept.
> Target-based IDS has two components, a correlation mechanism
> *and* a target-based IDS sensor, this article only reviews the former.
CTR was designed to assist in the post-attack investigation and response
phases of network attacks. We have no interest in correlation,
aggregation, etc. We are looking at each attack as a serious problem and
investigating it separately in that context. CTR requires no
target-based input from the IDS sensor other than the alert. In this
way, CTR emulates many of the steps an administrator may take if
investigating an attack.
For example:
1) A URL attack is seen by the sensor affecting Windows IIS.
2) A check is made if the attacked OS is Windows.
3) A *low-impact* external assessment is made if available.
4) If Windows, then log onto the host and check for relevant hotfixes,
patches, etc. for the vulnerability.
5) If the system is not patched, retrieve the log files and search for
successful signs of the attack.
6) If the attack traces are found in the log, then escalate the alert.
7) Administrator can view the chronological event log we generated of
each and every step we took to investigate the attack (from IDS
detection to final resolution and reason why we upgraded/downgraded).
8) Administrator can view the actual log files retrieved from the
impacted host to manually verify if the attack was successful or not.
(The above process happens in a very fast and automated fashion.)
Strictly speaking, CTR doesn't even need IDS alerts to do this activity.
Administrators can either schedule agents to periodically check for
problems, or it can be used to manually run investigations above and
beyond the detected attack. So even if an attack was downgraded, but
you're still suspicious of other problems, you can initiate further
investigations on the fly.
So in the definition of target IDS generally given you'd be able to only
accomplish steps 1-3 reliably (step 3 may not be low-impact) and perhaps
step 4 patch checks with certain Windows probes. However steps 5-8 are
totally out of the question. That's why we don't follow the traditional
targeted IDS philosophy. I think admins want more concrete data about
security incidents on their network and deeper automated investigations
are required to handle this task.
> For the benefit of the readers in this forum, I'll repeat
> myself from
> our
> exchange in November:
>
> "Additionally, since I came up with the term "Target-based
> IDS" I'd like to define the components of a true TIDS. TIDS is *not*
> event->vuln correlation, that's event contextualization (or impact
> assessment). We perform event contextualization so that we
> can reduce the number of events generated by a NIDS to a
> manageable amount, but it's only one leg of a full blown TIDS
> solution.
>
> There are three classes of problems in IDS that require us to
> transition to TIDS:
> 1) Lack of impact assessment/prioritization
> 2) Lack of host context (OS identification, service detection)
> 3) Lack of network context (topology discovery)
>
> Problem one stops us from getting use of the data generated
> by IDSes. The entire value of IDS is in its output, if we
> can't reduce that output to information that's useful to us
> as administrators then the usefulness of entire system is
> limited. Tenable and ISS [mfr: and Cisco] both have
> solutions to solve problem 1 and Sourcefire is working on one (RNA).
>
> Problems 2 and 3 are what Ptacek and Newsham were talking
> about. If an attacker can know more about the targets he's
> attacking than the IDS, he can use that knowledge to get
> around the IDS. If you're going to defeat that then you need
> to drive the host and network context into the IDS process
> itself, post-processing won't buy you anything if the IDS
> sensor isn't as accurate as possible. This is the *heart* of
> TIDS, you can't have a TIDS if you don't incorporate
> host/network context directly into the IDS process itself,
> the accuracy of the system will always be suspect and the 1st
> part of the triad will not be as useful as it should be."
I think you miss quite a bit of data just watching the wire and not
connecting to the host and see what is going on directly from within.
Some IDS signatures are just too generic to classify without looking.
For instance Cisco IDS has a signature that looks for "cmd.exe" being
executed in a URL (other IDS's have similar sigs). In this case the IDS
has no clue what the person was trying to do, all they saw was "cmd.exe"
pass in a URL to the server. If you're lucky you may get back a response
to the request that indicates it failed, but sometimes you don't. In
this case a vulnerability probe may not work because the attack doesn't
map to a specific problem per se and watching the return traffic could
be inconclusive. The only way to find this type of attack (and others
like it) is to actually connect to the system in question and scan the
log files for indicators of the attack. CTR addresses points 1 and 2
above and the way the dynamic process works it also makes things like
network context (point 3 above) much less relevant.
Targeting IDS data is a good idea and has its uses, but many admins
don't know what to do with the event even after it's been escalated.
This is why we are focusing on automated attack investigation and
response. Doing straight targeted IDS doesn't always provide the answers
people want and isn't capable of directly collecting forensic
information from the affected host for automated or manual
investigations. So perhaps the definition of targeted IDS should expand
to include actual on-host investigation techniques, or we can create a
new buzzword to encompass what it is that CTR actually does above
straight targeted-IDS.
I thought the article was fair in its comments with both positive and
negative points of the CTR product pointed out. We're working to correct
the issues brought out by Joel and are growing the investigative
capabilities to make more intelligent use of data we receive.
Comments inline...
> First, I find it troubling that the history and full meaning
> of the term "target-based IDS" (which I coined in 2000) was
> omitted. That this article didn't review any fully
> target-based IDS products will almost certainly leave readers
> with a misunderstanding of what target-based IDS really is.
Cisco Threat Response (CTR) was designed to be an automated forensic
investigation and response tool. An artifact of this process is that it
eliminates false alarms, but this has never been the exclusive thrust of
the product. So in this context CTR was never designed to be a
target-based IDS or even a facilitator of that concept.
> Target-based IDS has two components, a correlation mechanism
> *and* a target-based IDS sensor, this article only reviews the former.
CTR was designed to assist in the post-attack investigation and response
phases of network attacks. We have no interest in correlation,
aggregation, etc. We are looking at each attack as a serious problem and
investigating it separately in that context. CTR requires no
target-based input from the IDS sensor other than the alert. In this
way, CTR emulates many of the steps an administrator may take if
investigating an attack.
For example:
1) A URL attack is seen by the sensor affecting Windows IIS.
2) A check is made if the attacked OS is Windows.
3) A *low-impact* external assessment is made if available.
4) If Windows, then log onto the host and check for relevant hotfixes,
patches, etc. for the vulnerability.
5) If the system is not patched, retrieve the log files and search for
successful signs of the attack.
6) If the attack traces are found in the log, then escalate the alert.
7) Administrator can view the chronological event log we generated of
each and every step we took to investigate the attack (from IDS
detection to final resolution and reason why we upgraded/downgraded).
8) Administrator can view the actual log files retrieved from the
impacted host to manually verify if the attack was successful or not.
(The above process happens in a very fast and automated fashion.)
Strictly speaking, CTR doesn't even need IDS alerts to do this activity.
Administrators can either schedule agents to periodically check for
problems, or it can be used to manually run investigations above and
beyond the detected attack. So even if an attack was downgraded, but
you're still suspicious of other problems, you can initiate further
investigations on the fly.
So in the definition of target IDS generally given you'd be able to only
accomplish steps 1-3 reliably (step 3 may not be low-impact) and perhaps
step 4 patch checks with certain Windows probes. However steps 5-8 are
totally out of the question. That's why we don't follow the traditional
targeted IDS philosophy. I think admins want more concrete data about
security incidents on their network and deeper automated investigations
are required to handle this task.
> For the benefit of the readers in this forum, I'll repeat
> myself from
> our
> exchange in November:
>
> "Additionally, since I came up with the term "Target-based
> IDS" I'd like to define the components of a true TIDS. TIDS is *not*
> event->vuln correlation, that's event contextualization (or impact
> assessment). We perform event contextualization so that we
> can reduce the number of events generated by a NIDS to a
> manageable amount, but it's only one leg of a full blown TIDS
> solution.
>
> There are three classes of problems in IDS that require us to
> transition to TIDS:
> 1) Lack of impact assessment/prioritization
> 2) Lack of host context (OS identification, service detection)
> 3) Lack of network context (topology discovery)
>
> Problem one stops us from getting use of the data generated
> by IDSes. The entire value of IDS is in its output, if we
> can't reduce that output to information that's useful to us
> as administrators then the usefulness of entire system is
> limited. Tenable and ISS [mfr: and Cisco] both have
> solutions to solve problem 1 and Sourcefire is working on one (RNA).
>
> Problems 2 and 3 are what Ptacek and Newsham were talking
> about. If an attacker can know more about the targets he's
> attacking than the IDS, he can use that knowledge to get
> around the IDS. If you're going to defeat that then you need
> to drive the host and network context into the IDS process
> itself, post-processing won't buy you anything if the IDS
> sensor isn't as accurate as possible. This is the *heart* of
> TIDS, you can't have a TIDS if you don't incorporate
> host/network context directly into the IDS process itself,
> the accuracy of the system will always be suspect and the 1st
> part of the triad will not be as useful as it should be."
I think you miss quite a bit of data just watching the wire and not
connecting to the host and see what is going on directly from within.
Some IDS signatures are just too generic to classify without looking.
For instance Cisco IDS has a signature that looks for "cmd.exe" being
executed in a URL (other IDS's have similar sigs). In this case the IDS
has no clue what the person was trying to do, all they saw was "cmd.exe"
pass in a URL to the server. If you're lucky you may get back a response
to the request that indicates it failed, but sometimes you don't. In
this case a vulnerability probe may not work because the attack doesn't
map to a specific problem per se and watching the return traffic could
be inconclusive. The only way to find this type of attack (and others
like it) is to actually connect to the system in question and scan the
log files for indicators of the attack. CTR addresses points 1 and 2
above and the way the dynamic process works it also makes things like
network context (point 3 above) much less relevant.
Targeting IDS data is a good idea and has its uses, but many admins
don't know what to do with the event even after it's been escalated.
This is why we are focusing on automated attack investigation and
response. Doing straight targeted IDS doesn't always provide the answers
people want and isn't capable of directly collecting forensic
information from the affected host for automated or manual
investigations. So perhaps the definition of targeted IDS should expand
to include actual on-host investigation techniques, or we can create a
new buzzword to encompass what it is that CTR actually does above
straight targeted-IDS.
I thought the article was fair in its comments with both positive and
negative points of the CTR product pointed out. We're working to correct
the issues brought out by Joel and are growing the investigative
capabilities to make more intelligent use of data we receive.
-- Craig
------------------------------------------------------------------------
---
------------------------------------------------------------------------
---
[ reply ]