> Hi Craig,
>
> Very quickly, I'll touch on a couple things. I really don't want to
> turn this into a big thread if at all possible.
No problem. This will be my last post on this for a while. Discussions
like this are often only resolved by users deciding what approach suits
their needs the best.
> [snip]
>
> > For example:
> >
> > 1) A URL attack is seen by the sensor affecting Windows IIS.
> > 2) A check is made if the attacked OS is Windows.
> > 3) A *low-impact* external assessment is made if available.
> > 4) If Windows, then log onto the host and check for
> relevant hotfixes,
> > patches, etc. for the vulnerability.
> > 5) If the system is not patched, retrieve the log files and
> search for
> > successful signs of the attack.
> > 6) If the attack traces are found in the log, then escalate
> the alert.
> > 7) Administrator can view the chronological event log we
> generated of
> > each and every step we took to investigate the attack (from IDS
> > detection to final resolution and reason why we
> upgraded/downgraded).
> > 8) Administrator can view the actual log files retrieved from the
> > impacted host to manually verify if the attack was
> successful or not.
> >
> > (The above process happens in a very fast and automated fashion.)
> >
>
> [snip]
>
> > So in the definition of target IDS generally given you'd be able to
> > only
> > accomplish steps 1-3 reliably (step 3 may not be low-impact) and
> > perhaps
> > step 4 patch checks with certain Windows probes. However
> steps 5-8 are
> > totally out of the question. That's why we don't follow the
> traditional
> > targeted IDS philosophy. I think admins want more concrete
> data about
> > security incidents on their network and deeper automated
> investigations
> > are required to handle this task.
>
> The problem is that if you don't do prediscovery you can't trust the
> information you get back after the attack. Once the host is
> compromised you can't believe anything it tells you IMHO.
However both active and passive prediscovery fail in a number of cases:
1) On very large networks often scanning large numbers of hosts will
take too long and be too disruptive.
2) On networks that are very dynamic with a lot of DHCP, VPN users, or
people moving between wireless access points, the data becomes stale
quickly.
3) On systems where users made changes to the config that don't show up
on the network until attacked.
4) Background changes and updates to network systems through automated
patch distribution systems, etc.
5) ...
It's been our experience that most attacks/attackers leave traces on
systems that can be spotted even though the system itself may be
compromised. Even if the root attack can't be seen, the system
config/state often reveals it was likely the attack worked. Automating
the investigative process also gives administrators an advantage because
they rarely know everything to look for to validate detected attacks.
Heck, I've been doing this stuff for over a decade now and the last few
years have seen so many new attacks come out that I don't even pretend
to know them all any more, let alone what to look for on a host for each
and every one. That's why this problem needs automation in a big way. By
automating the process of investigation you bring consistency and
repeatability even if your admin staff have different levels of
experience dealing with these issues. With CTR we can say "We looked at
this box and it wasn't patched against this attack and here are the log
files from the system so you can see what was reported yourself."
Lastly, the gap left between a reported attack (even one that was
validated with prediscovery) and the time the admin does anything is
critical for response purposes. Every second, minute, hour, day, the
attacker is left alone on the computer without admin action is serious
business. CTR helps reduce this window of opportunity because we will,
when warranted, quickly grab system logs and other data from the host in
real-time. If the admin isn't around to do anything at the moment, at
least they have a set of data that wasn't laying on a host being edited
by the attacker to cover their trail. If we reported that file X was
present and patch X wasn't applied and then suddenly the admin looks and
it's gone and the system is patched, well they better be suspicious that
something is wrong.
> > I think you miss quite a bit of data just watching the wire and not
> > connecting to the host and see what is going on directly
> from within.
>
> How about in the case of welchia? The host gets popped and
> the vuln is
> patched in one shot, step 3 fails and the system is none the
> wiser. In
This was just one example. The checks may look for different issue
depending on what the attack is. So for instance on certain Trojan horse
activity we may check for known suspicious registry keys and dropper
files, etc. It varies.
Also Welchia leaves plenty of information around to see something
happened even if the system appears patched. In this particular case
though the worm would have had to download and apply the hotfix within
the 1-2 seconds it takes for CTR to connect to the host and look around.
In this instance we don't have a Welchia agent directly created because
this likelihood is pretty small and the detected signature applies to
multiple attack possibilities.
> the event of a successful overflow or other admin/root
> granting remote
> attacks, the configuration of the host after the attack may
> be nothing
> like what's required to elevate your level of suspicion.
Possibly and I'll go one further: As the CTR methodologies become more
ingrained in the Cisco architecture we're expecting attackers to adapt
their techniques to counter our current techniques, etc. The moment
attackers begin altering their methods to defeat CTR (or other targeted
IDS methods) is an indicator of success in the battle. You forced the
opponent to change strategy which happens all the time when effective
countermeasures are used. I would never (and I doubt you would) say that
counters won't evolve to bypass *any* security product, in fact
effective security means new attacks will emerge (and helps ensure our
job security). If attackers suddenly gave up one day it would make
things really boring and frankly I think many of these attacks are
clever even if they are keeping us all up over the weekends. When the
attacks change, so will CTR (and other solutions).
However, in the same vein an attacker could also have their attack
initiate a DHCP release/renew, grab a new IP address, and initiate an
encrypted back channel communication out to the attacker over the HTTP
SSL port. This would completely bypass data collected actively/passively
by a TIDS as the system has a new IP. When the admin went looking for
the affected host they may not find it out of hundreds (thousands) of
addresses in the DHCP pool. This is of course assuming the admin even
knows what to look for given the number of attacks that exist now. So
all approaches have their potential weaknesses...
> > This is why we are focusing on automated attack investigation and
> > response. Doing straight targeted IDS doesn't always provide the
> > answers people want and isn't capable of directly
> collecting forensic
> > information from the affected host for automated or manual
> > investigations.
>
> Automated forensics are useful and a nice step forward but if the
> attacker evades the IDS or disables the forensics systems
> once he's on
> the host then you've got a problem. The methodology described also
> reveals a lot of information about the security posture of the site
> that's being hit (i.e. Cisco CTR is in use) and what countermeasures
> are appropriate to reduce the chances of being detected in
> the future.
> In the worst case it could also be used to suggest what the
> configuration of the IDS is...
CTR doesn't have any presence on the host (we currently use ordinary
login credentials like any other user) so there isn't a local agent to
disable. This was a deliberate design to ensure the attacker has as
little knowledge of possible as to what protection the host has, what
CTR may look for, and how they may go about avoiding detection.
However, as pointed out above, I think all approaches have limitations
given the ingenuity of attackers. I'd expect administrators worth their
title to apply security solutions in layers for maximum effect. Doing
otherwise is a serious mistake. This probably means a combination of
products.
> > So perhaps the definition of targeted IDS should expand
> > to include actual on-host investigation techniques, or we
> can create a
> > new buzzword to encompass what it is that CTR actually does above
> > straight targeted-IDS.
>
> Automated network forensics? I think TIDS has a concise definition
> right now, I'm not in favor of extending it.
That's OK. I'll let the marketing guys come up with something. I'm not
really good at this stuff.
> Hi Craig,
>
> Very quickly, I'll touch on a couple things. I really don't want to
> turn this into a big thread if at all possible.
No problem. This will be my last post on this for a while. Discussions
like this are often only resolved by users deciding what approach suits
their needs the best.
> [snip]
>
> > For example:
> >
> > 1) A URL attack is seen by the sensor affecting Windows IIS.
> > 2) A check is made if the attacked OS is Windows.
> > 3) A *low-impact* external assessment is made if available.
> > 4) If Windows, then log onto the host and check for
> relevant hotfixes,
> > patches, etc. for the vulnerability.
> > 5) If the system is not patched, retrieve the log files and
> search for
> > successful signs of the attack.
> > 6) If the attack traces are found in the log, then escalate
> the alert.
> > 7) Administrator can view the chronological event log we
> generated of
> > each and every step we took to investigate the attack (from IDS
> > detection to final resolution and reason why we
> upgraded/downgraded).
> > 8) Administrator can view the actual log files retrieved from the
> > impacted host to manually verify if the attack was
> successful or not.
> >
> > (The above process happens in a very fast and automated fashion.)
> >
>
> [snip]
>
> > So in the definition of target IDS generally given you'd be able to
> > only
> > accomplish steps 1-3 reliably (step 3 may not be low-impact) and
> > perhaps
> > step 4 patch checks with certain Windows probes. However
> steps 5-8 are
> > totally out of the question. That's why we don't follow the
> traditional
> > targeted IDS philosophy. I think admins want more concrete
> data about
> > security incidents on their network and deeper automated
> investigations
> > are required to handle this task.
>
> The problem is that if you don't do prediscovery you can't trust the
> information you get back after the attack. Once the host is
> compromised you can't believe anything it tells you IMHO.
However both active and passive prediscovery fail in a number of cases:
1) On very large networks often scanning large numbers of hosts will
take too long and be too disruptive.
2) On networks that are very dynamic with a lot of DHCP, VPN users, or
people moving between wireless access points, the data becomes stale
quickly.
3) On systems where users made changes to the config that don't show up
on the network until attacked.
4) Background changes and updates to network systems through automated
patch distribution systems, etc.
5) ...
It's been our experience that most attacks/attackers leave traces on
systems that can be spotted even though the system itself may be
compromised. Even if the root attack can't be seen, the system
config/state often reveals it was likely the attack worked. Automating
the investigative process also gives administrators an advantage because
they rarely know everything to look for to validate detected attacks.
Heck, I've been doing this stuff for over a decade now and the last few
years have seen so many new attacks come out that I don't even pretend
to know them all any more, let alone what to look for on a host for each
and every one. That's why this problem needs automation in a big way. By
automating the process of investigation you bring consistency and
repeatability even if your admin staff have different levels of
experience dealing with these issues. With CTR we can say "We looked at
this box and it wasn't patched against this attack and here are the log
files from the system so you can see what was reported yourself."
Lastly, the gap left between a reported attack (even one that was
validated with prediscovery) and the time the admin does anything is
critical for response purposes. Every second, minute, hour, day, the
attacker is left alone on the computer without admin action is serious
business. CTR helps reduce this window of opportunity because we will,
when warranted, quickly grab system logs and other data from the host in
real-time. If the admin isn't around to do anything at the moment, at
least they have a set of data that wasn't laying on a host being edited
by the attacker to cover their trail. If we reported that file X was
present and patch X wasn't applied and then suddenly the admin looks and
it's gone and the system is patched, well they better be suspicious that
something is wrong.
> > I think you miss quite a bit of data just watching the wire and not
> > connecting to the host and see what is going on directly
> from within.
>
> How about in the case of welchia? The host gets popped and
> the vuln is
> patched in one shot, step 3 fails and the system is none the
> wiser. In
This was just one example. The checks may look for different issue
depending on what the attack is. So for instance on certain Trojan horse
activity we may check for known suspicious registry keys and dropper
files, etc. It varies.
Also Welchia leaves plenty of information around to see something
happened even if the system appears patched. In this particular case
though the worm would have had to download and apply the hotfix within
the 1-2 seconds it takes for CTR to connect to the host and look around.
In this instance we don't have a Welchia agent directly created because
this likelihood is pretty small and the detected signature applies to
multiple attack possibilities.
> the event of a successful overflow or other admin/root
> granting remote
> attacks, the configuration of the host after the attack may
> be nothing
> like what's required to elevate your level of suspicion.
Possibly and I'll go one further: As the CTR methodologies become more
ingrained in the Cisco architecture we're expecting attackers to adapt
their techniques to counter our current techniques, etc. The moment
attackers begin altering their methods to defeat CTR (or other targeted
IDS methods) is an indicator of success in the battle. You forced the
opponent to change strategy which happens all the time when effective
countermeasures are used. I would never (and I doubt you would) say that
counters won't evolve to bypass *any* security product, in fact
effective security means new attacks will emerge (and helps ensure our
job security). If attackers suddenly gave up one day it would make
things really boring and frankly I think many of these attacks are
clever even if they are keeping us all up over the weekends. When the
attacks change, so will CTR (and other solutions).
However, in the same vein an attacker could also have their attack
initiate a DHCP release/renew, grab a new IP address, and initiate an
encrypted back channel communication out to the attacker over the HTTP
SSL port. This would completely bypass data collected actively/passively
by a TIDS as the system has a new IP. When the admin went looking for
the affected host they may not find it out of hundreds (thousands) of
addresses in the DHCP pool. This is of course assuming the admin even
knows what to look for given the number of attacks that exist now. So
all approaches have their potential weaknesses...
> > This is why we are focusing on automated attack investigation and
> > response. Doing straight targeted IDS doesn't always provide the
> > answers people want and isn't capable of directly
> collecting forensic
> > information from the affected host for automated or manual
> > investigations.
>
> Automated forensics are useful and a nice step forward but if the
> attacker evades the IDS or disables the forensics systems
> once he's on
> the host then you've got a problem. The methodology described also
> reveals a lot of information about the security posture of the site
> that's being hit (i.e. Cisco CTR is in use) and what countermeasures
> are appropriate to reduce the chances of being detected in
> the future.
> In the worst case it could also be used to suggest what the
> configuration of the IDS is...
CTR doesn't have any presence on the host (we currently use ordinary
login credentials like any other user) so there isn't a local agent to
disable. This was a deliberate design to ensure the attacker has as
little knowledge of possible as to what protection the host has, what
CTR may look for, and how they may go about avoiding detection.
However, as pointed out above, I think all approaches have limitations
given the ingenuity of attackers. I'd expect administrators worth their
title to apply security solutions in layers for maximum effect. Doing
otherwise is a serious mistake. This probably means a combination of
products.
> > So perhaps the definition of targeted IDS should expand
> > to include actual on-host investigation techniques, or we
> can create a
> > new buzzword to encompass what it is that CTR actually does above
> > straight targeted-IDS.
>
> Automated network forensics? I think TIDS has a concise definition
> right now, I'm not in favor of extending it.
That's OK. I'll let the marketing guys come up with something. I'm not
really good at this stuff.
Thanks,
-- Craig
------------------------------------------------------------------------
---
------------------------------------------------------------------------
---
[ reply ]