, SecurityFocus 2000-09-04
Last month's Brown Orifice program opened a backdoor to an insecure future. Can open source save the day?
Because the new security flaws are so different, a different sort of programmer is likely to find them.
It's tempting to hop into the arena and vilify the software company responsible for turning out any product with a high-profile security bug as serious as this one, but in this case, at least, that would be myopic. Fault for this bug does not lie entirely with Netscape: any web browser with a Java Virtual Machine (JVM) is affected to some extent. Some of the code involved does not belong to Netscape but is instead a component they licensed from another vendor, Sun Microsystems.
Brown Orifice exists because the people who understood the different components did not or could not see how the interaction of these various pieces could cause trouble. The most dangerous, well-concealed, complex, and noteworthy security flaws in the future will be of this sort.
The Brown Orifice program exploits not just one area of functionality but two: the first in the integral part of universal Java functionality known as the Java Core, and the second in the JVM built into the browsers. Only someone intimately familiar with Java's language, the structure of the Java Core, and with the browser products themselves could have discovered this flaw. The difficulty of discovering such a hole is partly demonstrated by the fact that these existed undetected (or at least, unpublished) for so long. And when they did come out, it wasn't a member of Netscape's development team who found them; it was independent programmer Dan Brumleve.
Netscape's last major
Lest you suspect that Netscape has only recently fallen on hard times with exploits against their browser, remember that then-UC Berkeley students Ian Goldberg and David Wagner discovered the first major Netscape security flaw five years ago, in September 1995. By taking a look at how Netscape generated random numbers, they discovered that if you know when a connection was established then Netscape's encryption was useless.
In all three cases, the proximate causes of these holes may have differed, but the same formal cause undergirds them: the programmers failed to fully comprehend the interactions between the main body of code and various components produced by other people or groups and incorporated into the software. Like an otherwise fine suit with poor stitching at the sleeves, the software literally falls apart at the seams.
Netscape is far from the only development team to make this mistake: difficulties in securing component architecture are at the root of the spate of macro viruses--including last year's Melissa worm--on Microsoft's Office platform, as well as a number of problems with Internet Explorer's ActiveX components. In both cases, the failure of the engineers to secure the interface between components (Visual Basic in the first case, COM in the second) results in any element having the power to subvert any other element.
In the past, many security holes were problems in the software itself: the latest server module does not check the length of a string and allocates too much memory, or some new killer app does not check the permissions of the file, and, having too much privilege on the system, overwrites it.
This new class of holes is different: a programmer doesn't understand what somebody else was thinking or expecting, and fails to make a smooth transition between the two pieces of code. Sometimes the transition appears smooth, but the underlying complexity scuttles system security in the end.
The key to solving this problem is the open source movement, and its propensity for keeping code development simple and ego-free.
Raymond's formulation of Linus's Law in his classic open source polemic
Just as adding programmers to a project makes it later, so does it further divide both the understanding of the code base among the developers and the number of pieces out of which the program is built. In the open source world, the ability to reuse existing code and the freedom to examine and learn any piece of it increases the number of people who have the ability to check the joints between the components in terms of both the underlying knowledge of the system and the freedom to examine the code for flaws.
Because the new inter-component security flaws differ so substantially from more traditional holes, a different sort of programmer is likely to find them. Open source allows the widest variety of coders to search the source for the flaws that they know best. This can only improve security.
There is hope for Netscape's disjointed code, perhaps even in the near future. Netscape 6.0, due to be released soon, will be based on the code generated by the Mozilla project. Mozilla has enjoyed support from Netscape, Inc. but has essentially built its browser from scratch. Indications are strong that Mozilla will at least alleviate the worst security problems caused by component software. Over the last two years, a team of programmers who really seem to understand both the issues and the code base have coalesced around the project. The recent re-licensing of Mozilla code under the Gnu Public License can only encourage a community to form around the software, filling the human gaps between the blocks of code.