Story continued from Page 1
Do you think that counting the number and the severity of the advisories is the best way to value the security of a software?
Surprisingly (and yes, this is foreshadowing), there's actually something useful about these sorts of metrics. The security industry is fairly slow when it comes to adapting to new threat models. Now this is understandable -- business and engineering requirements make it quite expensive to defend against every new threat that might come down the pike, and there are a near-infinite number of potential threats. That being said, the very name of the "antivirus" industry is practically a shrine to antiquated threats, and the vendor response to the spyware explosion was... underwhelming. Anything that can be done to accelerate industry awareness of rapidly growing threats is very useful, and seeing a vuln count grow from zero to twenty in a quarter is a good way to recognize a growing threat.
Comparing a given trait with itself in the past -- this can help. But when vulnerability counts are examined across products and vendors -- this is when things go from vaguely useful to completely disastrous.
Recently, it was observed that during a given period of time, the motley crew of major security vendors suffered around twenty four advisories between them, while Microsoft's product lines suffered only twenty three during the same time period. The implication was that, contrary to expectation, users were actually slightly safer just with their stock machine than by... I don't know, installing every vendor's local security agents on top of their operating systems? Now, I'll grant that if someone was insane enough to do this, their system would indeed collapse of its own weight. It wouldn't even require an attacker's help to fail ignomiously, if someone were to deploy a system like this. But nobody would, and while network-facing security agents are a growing market, certainly they do not possess -- even between them -- the market share of Microsoft Windows itself. Equivocating the combined threat of relatively boutique security code with the individual threat of the world's most popular operating system is...well, it's the difference between a couple regional restaurant chains having a problem with e.coli, and McDonalds distributing bad food worldwide.
The problem, of course, is one of units. 1 can be far more than 100, when 1 is in millions and 100 is in pennies. Determining units for comparative analysis of security threats is actually a genuinely difficult problem. Not just one but two talks at the recent CanSecWest conference went in-depth into the mechanisms by which organizations can effectively compare, for prioritization purposes, the relevance of a particular issue. To be honest, at first I was confused why Cisco's Mike Schiffman -- author of the Libnet toolkit I based most of my packet-based software on -- was spending his time at CanSec speaking about CVSS, the Common Vulnerability Scoring System. Why wasn't he showing some new packet manipulation mechanisms? Because good metrics aren't just important -- they're missing, and we're suffering as an industry because of it. Case in point: What, precisely, does 'critical' mean? What predictive value is associated with such a determination? On what basis is such a determination made? And how much credibility do we associate with those that abuse the term?
Naive comparison is a real problem, and even things that seem to be "apples to apples" -- say, a comparison between the vendor-announced vulnerability counts of Microsoft Windows XP SP1 vs. Redhat Linux 9 -- fall apart the moment you compare what's in the box for both. XPSP1 ships with no databases, while Redhat ships with at least MySQL and PostgreSQL. Should Redhat be penalized for warning customers of potential problems that might be experienced on their platform, despite the fact that they didn't even write the software to begin with? When Oracle on XP has a vulnerability, Microsoft does not need to put out an advisory, instead Oracle does. Should Microsoft be referred to as a more secure platform because standard disclosure policies do not extend to announcing problems in software acquired entirely from a third party? Would Redhat suddenly become more secure if you had to download MySQL and PostgreSQL from their respective authors, with the requisite advisories coming from those authors and thus not counting from Redhat itself?
Bad metrics encourage bad decisions. Those that compare naively encourage naive security. It's 2005; it's a little late for that.
Microsoft is trying to force patches installation, and I'm wondering if you think that the auto-updater included in SP2 is enough. For example if they are really focused on security, why don't they require that each system that wants to access one of the various services such as Hotmail or MSN Messenger is updated? After all they have control on the OS, the IM client, and the browser...
Yes, because they own the only OS, the only IM client, and the only browser.
At the end of the day, your desktop is your property -- after all, are you or are you not the one liable for what the system does? (This is one of the things that ends DRM hardware in any corporate environment.) If you want to patch your system, it should be easy and silent and clean and elegant, which is a fairly good description of automatic updates in XP SP2. But if you don't want to -- what the heck, you want them to hold some kid's mail hostage because he didn't want to install a 20MB patchset? Respecting the owner of the hardware is critical, and your recommendations cross the line. MS may be respecting security, but they're respecting property rights too. This is good.
But some people keep getting exploited by spyware, viruses, and so on. Who should we blame then? OS vendors? Or the people who chose that particular software and then ran it without an appropriate secure setup?
Heh! You asked this to Marcus Ranum! :) Man had a point, though. It really is unfair to blame everyone but the people actually carrying out the attacks. This ain't Y2K and attackers are not some theoretical bogeyman -- systems that aren't secured are getting broken into on a regular basis, with real dollars being lost. People are doing this, and it's getting more professional, not less. (I'll return to this.)
Security engineers need to be really careful to keep their suggested solutions tractable -- it's one thing to demand a car that won't blow up when tapped on the rear bumper, but it's quite another thing to insist on a car that can't blow up, no matter what is done to it. The former is tractable, the latter is not. The only way to deal with the latter is to prevent people from wanting to sabotage / fire high explosives at vehicles in the first place.
Of course, the problem is that geographical constraints are far more powerful against car bombers than they are against network attackers. Physical attackers must be in short range of their target -- even those who use physical actions (such as skimming a credit card at a store) to effect international fraud are constrained by their body's location -- and, indeed, are fairly easy to hunt simply due to these constraints (find the common purchasing point of all defrauded parties, monitor [the] location for a few days, capture attacker). But such constraints really don't apply online. On the Internet, it's no harder to send a packet down the street than it is to send it 'round the world. We're rather blithe about the fact that the Internet erases borders and obviates geography, but for all the unquestionable good this yields -- the warlike anywhere can go on the warpath everywhere.
Now does this mean we go down some isolationist path, and fragment the global Internet into tiny national subnets? It has become difficult for many Nigerians to participate in the global email network, an upcoming talk at Defcon discusses blocking whole countries based on their IP address, and of course there's that pesky matter of Google delivering custom content based on whose borders you happen to be behind. So it's definitely something people have worked on. But to say something with full awareness of how ironic it truly is -- just because we can do something, doesn't mean we should. So much of the functional value of the Net comes from its sheer scale, and universal access. Security people tend to suffer some tunnel vision in recognizing what is valuable, functionally, about a given technology. For some, the only requirement is that it be secure -- speed, flexibility, usability, these are irrelevant, as long as the system is secure. Again, naive metrics lead to naive decision making. It may indeed turn out that creating a perfectly secure network is an intractable problem. Would this be a surprise? Has perfect security been found in anything? The cost of a free network may very well be to suffer those that abuse that freedom. To quote the obvious, "Eternal vigilance is the price of liberty."
That being said, it's important to recognize that accurate intelligence is a critical component of said vigilance. Marcus writes that we're being threatened by those who "place their desire for fun ahead of everyone on earth's desire for peace and the right for privacy". For the most part, this just isn't true anymore. Whoever stole those 40,000,000 credit card numbers wasn't someone testing a new exploit. The people extorting offshore gambling operations -- less "kid looking for fun", more "russian criminals running a protection racket". SPAM is money. Phishing is money. And Spyware -- certainly not a field populated by bored kids looking for another machine to own -- is lots and lots of money, and may very well be responsible for causing more damage than every other malware channel combined. Not that the kids have disappeared (and not that the slap on the Sasser author's wrist at all helps with that) -- it's just that, between the removal of low hanging fruit caused by XPSP2's firewall, and the rapidly growing population of professional black hats, the relevance of "kids just having fun" has diminished. There may of course be overlap -- kids grow up -- but it ain't fun and games anymore. We're spending billions to protect billions (or at least try to).
On the bright side, the more money is made by illicit behavior, the easier it becomes to track the thieves. Monetary flows are significantly less anonymous than IP packets.