Rootkits help hackers play hide-and-seek.
Many intruders are lazy or incompetent, and use old rootkits with known bugs in them.
I can't count the number of times I've received that answer after informing a system administrator that their system was the source an attack, and had probably been compromised.
I've been told that the system couldn't possibly be compromised, even when there was a root shell bound to port 60000, accessible to anyone with a copy of telnet on their system.
How can it be that a system administrator can't tell that a system has been hacked? More importantly, what can you do to find out if a particular system has been compromised?
It is nearly impossible to be certain that a system hasn't been compromised; if the system is online and running, and if the intruder was any good, it will be completely impossible to determine that a system has been hacked without first taking it offline.
This is in no small part due to the prevalence of "rootkits," replacement system binaries that hide the signs of a compromised system from its users.
For example, a rootkit may replace 'ps' with a version of the command that will not display information about particular processes, and may replace 'md5sum' with a version of the command that reports the expected --- though not accurate --- checksums for compromised system binaries. Other frequently-compromised binaries include ls, netstat, top; a relatively complete rootkit may include two dozen or more binaries, most of which are trojaned versions of standard system commands.
The standard approach to proving that a rootkit has been installed on a particular system is to boot the system from a known secure operating system install, such as a rescue CD or single-floppy Linux distribution, and (using a known-safe copy of md5sum) compare the checksums of system binaries to checksums from the genuine article.
Recently, conventional rootkits have begun supplanting 'kernel module rootkits', which are much more difficult to detect. But on systems compromised with conventional rootkits, comparison is still the best approach -- one made easier with the help of several utilities.
The chkrootkit tool, available from
If your system has not yet been compromised, you can use Tripwire (a free Linux version is available from
Oftentimes there are shortcuts to determining that a system was compromised. Many intruders are lazy or incompetent, and use old rootkits with known bugs in them. One of the more common Linux rootkits, for example, bases its compromised version of 'ps' on the older version of the command, which responds only to BSD-style flags, such as the common 'aux' combination. If you use a SYSV-style flag, such as '-ef', these versions of 'ps' will complain that those options are deprecated before returning some rather strange-looking output.
Another common rootkit problem is that 'netstat' will not respond to the '-plan' set of options, for reasons similar to those above. Of course, newer rootkits deal with these commands correctly, so correct behavior of these commands is not proof of an uncompromised system; incorrect behavior of these commands, however, is a strong indication of a compromised system.
Another good way to tell if a system has been compromised is to check to see if the ports that 'netstat' reports as listening exactly match the set of ports that 'nmap' -- a scanning tool -- reports when run from a remote system. A clever hacker will hide his backdoors with IP Chains or IP Tables, but oftentimes a backdoor will be globally accessible.
In general, if you can show discrepancies between the system's behavior and its expected behavior, you may have a hacked system. (Then again, you may just have found a bug.)
If the system's behavior has changed recently without any action on your part, that's quite likely a hacker.