Digg this story   Add to del.icio.us  
Falling Apart at the Seams
Kathleen Ellis and Jon Lasser, SecurityFocus 2000-09-04

Last month's Brown Orifice program opened a backdoor to an insecure future. Can open source save the day?

Brown Orifice. Not a pretty name, is it? As SecurityFocus.com reported last month, Brown Orifice can allow anyone on the Internet to browse your files. It affects you if you're a Windows, Macintosh, or Unix user who uses Netscape Navigator version 4.0 or later. The major security flaws it exploits have been lurking in your browser for well over two years, and could affect as many as three hundred million users worldwide.

It's tempting to hop into the arena and vilify the software company responsible for turning out any product with a high-profile security bug as serious as this one, but in this case, at least, that would be myopic. Fault for this bug does not lie entirely with Netscape: any web browser with a Java Virtual Machine (JVM) is affected to some extent. Some of the code involved does not belong to Netscape but is instead a component they licensed from another vendor, Sun Microsystems.

Brown Orifice exists because the people who understood the different components did not or could not see how the interaction of these various pieces could cause trouble. The most dangerous, well-concealed, complex, and noteworthy security flaws in the future will be of this sort.

The Brown Orifice program exploits not just one area of functionality but two: the first in the integral part of universal Java functionality known as the Java Core, and the second in the JVM built into the browsers. Only someone intimately familiar with Java's language, the structure of the Java Core, and with the browser products themselves could have discovered this flaw. The difficulty of discovering such a hole is partly demonstrated by the fact that these existed undetected (or at least, unpublished) for so long. And when they did come out, it wasn't a member of Netscape's development team who found them; it was independent programmer Dan Brumleve.

Netscape's History
Netscape's last major security flaw, discovered only weeks, ago was a buffer overflow accessible through comments embedded in JPEG-encoded graphics. Although the standard JPEG library discards the comment fields while processing the pictures, Netscape added code to retain the comments. This code failed to properly check the length of the comment field, which opened a major security hole that allowed the execution of arbitrary, and potentially malicious, code on some platforms.

Lest you suspect that Netscape has only recently fallen on hard times with exploits against their browser, remember that then-UC Berkeley students Ian Goldberg and David Wagner discovered the first major Netscape security flaw five years ago, in September 1995. By taking a look at how Netscape generated random numbers, they discovered that if you know when a connection was established then Netscape's encryption was useless.

In all three cases, the proximate causes of these holes may have differed, but the same formal cause undergirds them: the programmers failed to fully comprehend the interactions between the main body of code and various components produced by other people or groups and incorporated into the software. Like an otherwise fine suit with poor stitching at the sleeves, the software literally falls apart at the seams.

The Solution: Ego-free Software
Netscape is far from the only development team to make this mistake: difficulties in securing component architecture are at the root of the spate of macro viruses--including last year's Melissa worm--on Microsoft's Office platform, as well as a number of problems with Internet Explorer's ActiveX components. In both cases, the failure of the engineers to secure the interface between components (Visual Basic in the first case, COM in the second) results in any element having the power to subvert any other element.

In the past, many security holes were problems in the software itself: the latest server module does not check the length of a string and allocates too much memory, or some new killer app does not check the permissions of the file, and, having too much privilege on the system, overwrites it.

This new class of holes is different: a programmer doesn't understand what somebody else was thinking or expecting, and fails to make a smooth transition between the two pieces of code. Sometimes the transition appears smooth, but the underlying complexity scuttles system security in the end.

The key to solving this problem is the open source movement, and its propensity for keeping code development simple and ego-free.

Raymond's formulation of Linus's Law in his classic open source polemic The Cathedral and the Bazaar, that "Given enough eyeballs, all bugs are shallow," hints at the solution. Raymond also suggests that open source software need not fall prey to Brooks' Law, the belief that (in Raymond's words) "the complexity and communication costs of a project rise with the square of the number of developers, while work done only rises linearly." Raymond invokes Gerald Weinberg when adding, "in shops where developers are not territorial about their code, and encourage other people to look for bugs and potential improvements in it, improvement happens dramatically faster than elsewhere."

Just as adding programmers to a project makes it later, so does it further divide both the understanding of the code base among the developers and the number of pieces out of which the program is built. In the open source world, the ability to reuse existing code and the freedom to examine and learn any piece of it increases the number of people who have the ability to check the joints between the components in terms of both the underlying knowledge of the system and the freedom to examine the code for flaws.

Because the new inter-component security flaws differ so substantially from more traditional holes, a different sort of programmer is likely to find them. Open source allows the widest variety of coders to search the source for the flaws that they know best. This can only improve security.

There is hope for Netscape's disjointed code, perhaps even in the near future. Netscape 6.0, due to be released soon, will be based on the code generated by the Mozilla project. Mozilla has enjoyed support from Netscape, Inc. but has essentially built its browser from scratch. Indications are strong that Mozilla will at least alleviate the worst security problems caused by component software. Over the last two years, a team of programmers who really seem to understand both the issues and the code base have coalesced around the project. The recent re-licensing of Mozilla code under the Gnu Public License can only encourage a community to form around the software, filling the human gaps between the blocks of code.

    Digg this story   Add to del.icio.us  
Comments Mode:
User: friend or enemy? 2000-09-05
Pavel Roskin (1 replies)
User: friend or enemy? 2000-09-08
Open Source Solution 2000-09-05
Pete Kofod (2 replies)
Re: Open Source Solution 2000-09-08
angel'o'sphere (1 replies)
Re: Open Source Solution 2000-09-08
Open Source Solution 2000-09-08
Mike Crist
Open source is not a silver bullet 2000-09-06
Your friendly neighborhood software developer
"All bugs are shallow" is a delusion of Open Source Arguments 2000-09-08
peter (at) smalltalk (dot) org [email concealed]
Open source WORKS! 2000-09-08
Another friendly software developer
Mozilla and JavaScript 2000-09-08
Markus Fleck
How many ways can one article be wrong? 2000-09-08
Charles Miller


Privacy Statement
Copyright 2010, SecurityFocus