One of the recurrent questions in security is which product is the most secure. Windows Xp or Linux Debian ? Firefox or Internet Explorer ? Often theses comparisons are based on subjective opinions or the number of vulnerabilities count. A third way exist: Attack Surface analysis.
Often as a security specialist, my friends and students keep ask me which product is the more secure. That is a tough question, really because how do you measure a product security ?
Current Comparison Methodologies
Authoritative answer
Of course you can go for the authoritative proof : X (very famous) has said it. However science is about questioning and experiment so every time I heard this type of answer I wonder how this X guy knows that this product is more secure.
Statistical Analysis
The other well known method is to count the vulnerability that each product have suffers from the past. This is a method based on statistics and at the first sight it seems more reasonable. For example here is result of Nilotpal’s study to compare Vista to ubuntu Drapper. (You have a similar study that compare OSX and Vista on Larry’s zdnet blog)
While more scientific this methodology still have several flaws that make it quite unreliable:
First it assumes that from the past, you can predict the future. The basis of this approach is : if Y vulnerabilities have been discovered over the last X month then it is probable that same order of amount will be discovered in the next X month. If you use statistics on a subject that have pattern, it works really great. For example it works great for temperature, river water level or shop sales because they have pattern and cycle. For example temperature has a pattern conditioned by the rotation of earth over the sun and therefore you have a clear 12 months long cycle. But for system security there is no such cycle.
Moreover the only cycle that is well known for software is the product cycle : The interest for a product decreases with the time and ultimately it going to be supercede by a new version or a new one. It is the same for vulnerability analysis. When a product is released many peoples focus on find its holes. As time flows the number of people interested in finding holes might decrease as people move to other products/version.
The second flaw in the approach is that in regular statistical analysis you are able to say. Over X customer/month Y have been this or that. Here you don’t actually now how many people have look at the code to find holes and how much time they spend on it. So maybe there is more flaw found because more people are looking at this precise product.
Thirdly, a more subtle flaw in the approach appears when two products are compared over the last X month. This is not a faire comparison because theses their have been released at a different moment and therefore are not in the same part of their lifecyle. How comparing a product in its maturity stage against one in its introduction state can be objective ?
That is why, we need a more objective measure to compare products. Something that doesn’t rely on the past or some oracle but on facts. This is attack surface.
Attack Surface
An attack surface is a relative measure of product security. We say it is relative because it exists only in comparison to other products. For instance a spoon can be view as small only because some there is some bigger spoon: this is a relative measure. Similarly a product is more secure than an other (relative).
Absolute versus relative measure
An absolute measure is not possible because we can’t prove that a product is absolutely safe or has no bug. This is related to the halting machine problem and the rice theorem. If you are interested in bug detection take a look at ASTREE, the static analyzer made at the ENS.
The ultimate goal to achieve with attack surface is to be able to say “product X is more safe than product Y because it have a smaller attack surface”
Intuitively the attack surface aims at measuring how many attack vectors are available for each product. It does not measure if theses entries are actually used as attack vector but evaluate the potential. A way to view this is to think about mountain climbing: A way is relatively more easy than an other because it has more hooks to put your feet and hands. It does not tell you that the more difficult one is unusable or not it just tell you that it is more likely that easiest one is usable to reach your goal. Same for products, a product with a larger attack surface will be more likely vulnerable than one with a smaller one.
Attack surface history
Attack surface have been around in research since 2003. I believe that it is Michael Howard of Microsoft that informally define the notion of Relative Attack Surface Quotient (RASQ). The first paper on the subject called “Measuring Relative Attack Surfaces” was published in workshop by Michael Howard, Jon Pins, and Jeannette M. Wing in Dec. 2003. Since then the “Attack Surface Measurement” project is held at Carnegie Mellon.
How to measure it ?
So how an attack surface is measured ? Well that is the big challenge ! There is several on-going work on it but the basic idea is quite the same in every research.
You have three parts that define the attack surface :
- Target
- Enabler
- Vector
Targets are attacker objectives : a root shell is the most obvious one. Leaking sensitive data is an other etc …
Enablers are the set of process and services that allow the attacker to reach is goal for example an http server running.
Vectors can be view as the medium used to reach enabler and target. It can be a socket, memory sharing, pipe …
So roughly an attack surface is somehow the product of : Target X Enabler X Vector (It is not totally accurate and depends on the formalism but it should give you the idea). Stephen Northcutt intuitive definition is very bright (check his post on attack surface):
“We can define attack surface as our exposure, the reachable and exploitable vulnerabilities that we have. The best word picture I know of is the depiction of the Spartan Phlanx depicted in Warner Brothers’ tale of the Battle of Thermopylae, based on Frank Miller’s ‘300′.”
Some other criteria can be used to derive attack graph surface for instance you can use the LOC : Line of Code index. The idea behind this index is : the more the line of code there is the more likely their is bug. However this rule of thumb have also counter example (so far). For instance the iphone bootloader code is smaller that the baseband code. however bugs have been only found in the bootloader so far.
Current uses
Some product are already on the market for attack surface analysis such as Holodeck :
And you can find pretty good attack surface analysis that try to evaluate the potential security of product. The most famous is probably the “Windows Vista Network Attack Surface Analysis” by Symantec.
You also find a mention of attack surface in many windows 2008 preview. Such as in zdnet, 4sysops, and a windows2008blog.
Conclusion
Attack surface is currently the most scientificaly grounded method to compare product security. It is intuitive and simple in the concept but very complex to model and implement. This metric helps to answers important question such as does the new version of product X is better that Y from a security perspective.
See you next Sunday and a happy new year to you !
Latest Comments