Top-10 Vulnerability Discoverers of All Time (as well as 2008) - Who's in Pole Position?

Posted by Gunter Ollmann on February 17, 2009 at 4:26 PM EST.

Who discovers the most security vulnerabilities? That’s one of the more frequent questions I’ve encountered over the past few years. Funnily enough there’s usually a high correlation between the timing of my being asked and the latest marketing blitzkrieg customers may have encountered (not from IBM of course). It seems that every major (and not-so-major) security vendor goes though a phase of extolling the virtues of discovering their own subset of global security vulnerabilities – which further muddies the water.

So, who’s discovering the most security vulnerabilities? You can take a guess, but you’re probably wrong.

Since its conception, X-Force has been tasked with recording and analyzing all publicly disclosed security vulnerabilities. With around 40,000 disclosures catalogued thus far, the X-Force vulnerability database contains the wealth of well over a decade of threat analysis and plays a key role X-Force’s ability to understand the evolving threat landscape.

Note: If you get a chance to browse through the X-Force 2008 Trend & Risk Report (all 106 pages of it), you’ll get a better appreciation of the value X-Force extracts from the historical data.

One thing you won’t find in the report is the answer to the question though.

By way of history, a handful of years ago several large security vendors were jockeying for first place on the vulnerability discovery podium. With all the antics over which vendor was discovering the "most" (measured by values such as volume, criticality, "wormability" or even "coolness"), many security customers were missing the big picture – commercial vulnerability research groups only really discover a tiny fraction of the badness out there by themselves.

Things have changed a little since then – largely due to concerted efforts of the X-Force – and customers now have a much better grasp of the true nature of vulnerability discovery and its scale. With that in mind, I thought it was about time I shed a little light as to who the “shining stars” of vulnerability discovery are (or were).

Top Vulnerability Discoverers of All Time

Thanks to the X-Force Vulnerability Database team and their diligent analysis and cataloguing of forty-thousand public vulnerability disclosures, I can at last reveal who the top vulnerability discoverers are.

But before I do that I want to point out that the top-10 vulnerability discoverers of all time (running through to the end of 2008) account for approximately nine percent of all public disclosures, while 67 percent were found by other named individuals (with 8,670 different names), and the remaining 24 percent are attributed to unknown or anonymous researchers.

Which vulnerability researcher claims pole position for the most public vulnerability disclosures? The first-place title goes to Luigi Auriemma with 612 public disclosures. Up until 2008, the leader had been an individual using the handle “r0t” – but r0t appears to have ‘retired’ recently, and Luigi has since taken the prime spot.

Top-10 All-time Vulnerability Discoverers

Luigi describes himself as a 28 year old atheist living close to Milan in Italy, while r0t describes himself as being 18 years old and having defaced his first Web site when he was 10. Go figure.

Top Vulnerability Discoverers of 2008

The question you’re no doubt thinking of asking next is “who discovered the most vulnerabilities last year?” The title of top vulnerability discoverer for 2008 goes to Luigi Auriemma.

Top-10 2008 Vulnerability Discoverers

Looking at both the Top-10 All-Time and the Top-10 2008 lists, you’ll notice that handles are still in popular use by vulnerability researchers.

Thoughts and Caveats

The problem with constructing Top-10 lists like the two above is that you’re guaranteed to upset some vulnerability researcher out there that didn’t make it on to the list. So here are a few caveats to bear in mind:

1. While X-Force aims to analyze and catalogue every public vulnerability disclosure, it’s entirely possible that some vulnerabilities may be missed because they were disclosed on non-public lists or couldn’t be verified as actually being a real vulnerability.
2. In some cases a vulnerability may be publicly disclosed without an indicator as to who initially discovered the vulnerability (which is often intentionally done by the discoverer) – therefore we catalogue the researcher as “unknown”.
3. There are no points for sexiness of the vulnerability. The lists above are based purely on the number of public disclosures – not the quality of the vulnerability. Cross-site scripting vulnerabilities in a commercial shrink-wrapped application count for the same as a remote root vulnerability on a default Windows service.
4. If a researcher didn’t publicly disclose the vulnerability – it doesn’t count. So, even if you are an uber pentester that regularly discovers 100 new vulnerabilities with each consulting engagement, unless you released a public disclosure of the vulnerability – you weren’t counted (sorry).

Finally, I want to offer a special thanks to Brad Sherrill of the X-Force Vulnerability Database team for taking the time to extract the right data and uncovering the Who’s Who of vulnerability disclosure rankings.

Updates – 2/18/09

I’ve had a few responses to the blog entry. As expected, some people have a problem with the fact that it’s unfair to not factor in the “quality” of the vulnerability. True – but the analysis was focused on “who discovers the most vulnerabilities” not who was the “best” vulnerability researcher. I have some ideas on metrics/calculations for doing something like that, but it’s guaranteed that someone would have trouble with that too.

A couple of individuals wanted to point out that ‘rgod’ (ranked 3rd in the all-time list) actually passed away in early 2008 and that friends of his are maintaining his disclosure site. You can find information here and here.

Some researchers thought that when I stated "...be missed because they were disclosed on non-public lists" that ISS relied exclusively on BugTraq and Full-disclosure. Actually, that’s not the case – not by a long shot. ISS monitors many lists and research/disclosure forums, along with all the popular software vendor sites (basically, if you develop commercial software and have had a vulnerability disclosed publicly in the past, you’re added to the watch list going forward), as well as all the other disclosure sites (e.g. Secunia, OSVDB, milw0rm, etc.). I actually know of some research that’ll be coming public later this year (from an independent academic researcher) that compares the major vulnerability databases for depth of vulnerability coverage and speed of alerting – and that’ll clear up some things for the security industry.

Finally, there have been a few responses relating to the count of vulnerabilities attributed to a researcher. I think the various vulnerability database repositories out there do things slightly different. For example, here are a few things that the X-Force does:

1. The X-Force team researches each vulnerability disclosure to verify whether it is unique (i.e. not “rediscovered” by someone else). As you would expect, this can be time consuming, but needs to be done.
2. Each vulnerability is researched to verify whether it is actually a real vulnerability – along with the vendors response.
3. At any later point in time, if a vulnerability is found to be incorrect (i.e. not a vulnerability, or was previously discovered or covered by an earlier vulnerability disclosure) it will be “obsoleted” – which basically means it’ll still be recorded in the X-Force vulnerability database and available for internal viewing – but will not ever be included in any “real” vulnerability metrics or visible to the public.
4. Multiple vulnerabilities affecting the same product, the same version, and through the same vector, but with different parameters will be counted as a single vulnerability by X-Force (since they will require the same remediation and/or protection). For example, if an XSS vulnerability was announced in Product X affecting the a.php script and exploitable using the a, b, or c parameter, this would be broken up into 3 separate vulnerability entries in OSVDB, but only 1 in the X-Force DB.
5. The X-Force team are constantly updating the database records. Each vulnerability writeup also includes information about the vendors remediation and patch versions/links, as well as when/where exploit material has been observed, and common reverence codes such as BugID, CVE, etc. What this means is that over time vulnerability information is enhanced - and discoverers may change (e.g. a researcher may disclose 10 vulnerabilities under a couple of aliases, but later on in his "career" he decides to disclose new vulnerabilities under his own name - but explains that he used to use certain aliases in the past. Once known (and verified), the "discoverer" details of the old vulnerabilities are updated with the new information).