In the most recent example, a paper on automatic patch-based exploit generation (APEG) by some researchers at Carnegie Mellon and University of Pittsburgh contained some decent technical work, but put forth vastly overreaching conclusions that the sky was falling and massive technical measures must be undertaken to save the Internet. The research translated a program to a specialized intermediate language that can then be turned into an equation. This equation is solved to find what kind of data you need to send the program to reach a certain state. The technique was tested against a few sample vulnerabilities and demonstrated to crash the programs when given the patch information.

If you've listened to Halvar Flake at BlackHat for the past six years, this will sound eerily familiar. It's all very cool stuff technically and potentially useful for many things, finding zero-day vulnerabilities chief among them.

One issue an exploit developer may notice immediately while reading the paper is that crashing a program is not that same as exploiting it. Even having control of the instruction pointer is not even close to exploiting it in this day and age. In the paper, the authors stated that:

determining the specific address for a successful control hijack requires predicting the processes memory layout, which changes each time the process is invoked. Attackers currently do this by essentially repeatedly launching an attack until the memory layout matches what the exploit expects. We similarly repeatedly launch the attack until we achieve a successful control hijack.

An exploit in standard parlance is a program that can get control of another program, not just crash it. Crashing a program is known as producing a proof-of-concept. It's the difference between screaming "Haberdashery!" at someone until they go away and convincing them with reasoned argument.

As an aside, this is also why many academics have failed to understand why the security vulnerabilities market seems depressed. Writing a reliable exploit can take a long time even when you know about a vulnerability. It's always unknown if writing the exploit is even possible, until you have it working. If you buy a vulnerability for $10,000, you are expecting to potentially put another $80,000 into making it an exploit worth using. Of course, you wouldn't know this unless you had experience writing exploits.

The last paragraph of the APEG paper, a broad call to action, is summarized on the author's page as:

Attackers can simply wait for a patch to be released, use these techniques, and with reasonable chance, produce a working exploit within seconds. Coupled with a worm, all vulnerable hosts could be compromised before most are even aware a patch is available, let alone download it. Thus, Microsoft should redesign Windows Update.

First of all, every major security company has people who's job it is to figure out security vulnerabilities from patches. This is not a process that is unknown to the community. They've nearly always have been able to produce proof-of-concepts or exploits on the same day a patch has come out. In some cases, this is because they knew about the patch or vulnerability ahead of time, but in other cases, it's because the state of the art -- human analysis plus Zynamics BinDiff, fuzzers and specialized debuggers such as Immunity Debugger -- is currently good enough to provide quick response to patches.

This means that right now for vulnerabilities that are easy to turn into exploits, all vulnerable and reachable hosts can already be compromised before the automated Microsoft update process gets a chance to run. In the commercial world every patch must be tested thoroughly. Patching a large enterprise's machines within two weeks is considered lightning fast, far slower then the exploit development cycle. Changing Microsoft Update isn't going to change this.

The difference is that now there are few new vulnerabilities that are easy to turn from proof-of-concepts into exploits. Microsoft is largely responsible for making Windows's vulnerabilities harder to exploit when it shifted its strategy to focus on creating more secure software via the Trustworthy Computing Initiative. Turning on stack protection plus heap protection plus data-execution protection by default makes many vulnerabilities impossible to turn into exploits even with a proof-of-concept already done.

Thus, the majority of the vulnerabilities take much longer - perhaps months - to turn into exploits, assuming the vulnerabilities are even reachable in the default state through the default-enabled firewall. The notable exception to the Trustworthy Computing Intiative's general success has been Internet Explorer, which by definition has an extremely broad level of exposure.

Linux exploits, of course, are on average even harder to write than Windows exploits. Defeating GRSecurity, execshield, and modern compiler protections is not for the faint at heart or the cheap at wallet. Even doing exploit quality assurance against all the different flavors of Linux is prohibitively expensive.

Various people who do research for a living in the security industry have noted this disparity between the APEG paper's technical results and its conclusion. They often point out the strange statements in the paper itself, such as the lack of understanding as to what constitutes an exploit and how heap overflows are written. These sorts of misunderstandings are common among people who have not written exploits, the basic shibboleth of our community.

In the academic security environment, where there are social restrictions against writing exploits, you can find many examples of intellectual stagnation held up as valuable research. There is of course the canonical example of the DARPA-funded Gemini paper which protected against all stack overflows by making them heap overflows. In a modern environment such as Windows Server 2003 with Visual Studio's /GS protections, this bizarre technique turns a very difficult to exploit stack overflow into a much more easily (pdf) exploited heap overflow.

Like many people in the security community, I've walked into meetings with deans of computer science departments at well respected universities who want to solve the "firewall problem" by "inventing" firewalls that automatically reboot and clear state every so often. And, of course, there are papers on the economics of vulnerability disclosure, which assume that you can develop -- with math, no less -- a social policy on vulnerability disclosure that is enforceable or somehow valid.

There's great security technology that comes from university research programs. Yet, when it comes to making broad judgments about information security, such as how Windows Update should work, academic environments often lack the context and social background to be relevant.

That's why people who write papers in LaTeX two-column format end up saying the sky has a high negative trajectory, while the rest of us wish they'd stop living in the clouds.