Implementation of the preprocessor

Let's get into some implementation details. This section outlines the format of a typical Snort preprocessor and describes what the methods do. Pseudo code and method headers should help the reader understand the preprocessor internals. For more detail, you can download the preprocessor source code, which is a work in progress.

The header file in most preprocessors is very simple. You can see that the header in ours contains one function prototype:

extern void SetupIdentRogueHTTP();

The function itself is contained in the source file and its purpose will be discussed shortly. However, it has to be there in order for Snort to register the preprocessor.

Like any C code, the preprocessor source file starts with a variety of include statements. Some of the includes are specific to Snort development like Snort.h and plugbase.h. In general, it is not easy to code from scratch. It is much more efficient to just use all the same headers that other modules use (especially when dealing with an open source application) and figure out if you need them later. Also, you can always run an Internet search for more information on headers.

There are two major function prototypes. One is the init function:

extern void IdentRogueHttpInit(u_char *);

The other is the function that will process each packet:

extern void checkHttp(Packet *);

The names of the functions don't really matter all that much, so long as the rest of the code calls them properly. The init method should, of course, contain the word "init" at the end.

The init method is meant to perform three tasks: parse the arguments being sent in via the character pointer argument to the method, take any last minute setup steps on data structure being used in the preprocessor, and finally, link the preprocessor function into a list of functions. In our preprocessor example, a call is made to:

readServerHistory(args);

The method just reads in all of the information that the IDS already knows about the network Web servers. Then, it calls:

addFuncToPreprocList(checkHttp);

The name "method" pretty much says it all. It's adding the method that is going to be used to parse the packets to the preprocessor list. The method named in the argument will be called each time a packet is intercepted.

The method checkHttp is basically the brains of the preprocessor. You can see from the source code and the method prototype that it accepts a pointer to a packet structure as an argument. Below is some pseudo code that describes the example preprocessor. Be prepared--the claim is that this is intelligent--but, it just looks like a bunch of if else statements:

if(packet not TCP
     return
	
if(srcport not 80 and destport not 80)
     return

if(destport is 80 and destIP on local network)
     if(destIP is a known Web server)       
     # we expect this activity, so exit.
     if(destIP is not a known Web server)   
     # we're going to track this.

if(desport is 80 and destIP is not on local network)
     Our client browsing the Internet

if(the requests are unusually large)        
     # This is an anomaly, learn about it by noting it
     # We use our buffered http response sizes for comparison

if(srcport is 80 and srcIP is on local network)
     if(srcIP is a known Web server)
          # we could track packet size, but should probably just exit
     if(srcIP is not a known Web server)
          # then it's weird -- track it and learn if this is 
          # a new Web server or an anomaly

if(srcport is 80 and srcIP is not on local network)
     # buffer these to compare them with the GET requests 
     # being sent by the clients on the network

It doesn't seem very complex, but it shouldn't, for two reasons. First, pseudo code is meant to be easier to follow than the real code anyway-it's that transition step from idea to C code. Secondly, this is a proof of concept, but hopefully it has your gears churning, figuring out how you could expand on this idea. The important thing to remember is that the system is observing traffic and learning to change its behavior, based on the presence of Web traffic from IPs that aren't supposed to be Web servers.

Let's consider for a moment what kind of Snort rules you could write to handle the same kind of activity. There should be a rule for each of four scenarios. The first line of the Snort rules would be something like:

• alert TCP $HOME_NET any -> any 80
• alert TCP $HOME_NET 80 -> any any
• alert TCP $HOME_NET any <- any 80
• alert TCP $HOME_NET any <- any any

Right off the bat, there is an issue. IP addresses on our home net should be considered; however, machines that are already in the variable $ WEB_SERVERS should not raise any concerns. Let's take the first rule and elaborate on it:

alert TCP $HOME_NET any > any 80 (msg: "Large http request";
flow:to_server,established; dsize > $SOME_VARIABLE; sid: xxx; rev: 1;)

Well, that's pretty close to the desired result. Traffic from the network out to an HTTP server where the traffic is larger than expected. How large? Larger than some predefined variable. This rule can adapt-somewhat. The IT security specialist has to determine a value for $SOME_VARIABLE and adjust it as necessary. Also he or she would have to accommodate cases where a specific client legitimately performs that sort of request. For example, if a client tends to use HTTP as a mechanism for uploading pictures to a server, the IDS should be made aware of that. The authors don't claim to know all the tricks of Snort rule making, but if you want to maintain state information and learn dynamically from specific traffic packets, more is needed to support the rules. The learning preprocessor alleviates some of the work required by the IT security specialist to achieve the same goal.

Arguably, you could devise a series of Snort rules that could deal with the above situations, using some pretty complex alerting systems. Note that with this article, the whole goal is to demonstrate how you could introduce additional intelligence into your IDS and allow it to adapt to the network. This situation had been chosen because it is relatively simple and a decent solution, in the form of a preprocessor, can be explained clearly.

Conclusion

So, hopefully this article has perked up your interest in regards to making an IDS smarter using a preprocessor. This article really only scratched the surface of how artificial intelligence can lend a hand with making your IDS smarter. It also focused a lot on preprocessors in Snort as a method to implement more intelligence. The intelligence proposed here consisted of modeling machines in the network to determine if they were Web servers performing a legitimate service, as well as checking to see if clients were producing anomalous Web requests. A lot of research has been performed on artificial intelligence. For example, the work of Andres Arboleda and Charles Bedon (under the direction of Siler Amador) produced a preprocessor using neural networks in order to identity port scan traffic.

Another example is the work of Stefanos Koutsoutos, Ioannis Christou and Sofoklis Efremidis. They are working on an IDS for network-initiated attacks using a hybrid neural network. Also Gunes Kayacik, Nur Zincir-Heywood and Maclolm Heywood from Dalhousie University are doing a great deal of work on genetic programming and its use in anomaly detection. These are just a few examples and there are plenty more.

Preprocessors are a powerful feature of Snort and they have a lot of potential for increasing the intelligence of your IDS. Hopefully, by understanding the basics behind some rudimentary AI concepts and some of the advanced features of Snort, you will be able to think of new approaches for increasing intelligence in your IDS. Or, if you are a coder, experiment with your own preprocessors.

That concludes our paper on artificial intelligence and the impact it could have on an intrusion detection system to fulfill its mandate. We hoped you enjoyed the article and welcome your feedback.