Could you introduce yourself?

Pete Herzog: I'm Pete Herzog, Managing Director of ISECOM. I live in a small town in Catalonia just outside of Barcelona. It's also where I work part of the year. The other part of the year I work in the U.S. ISECOM is a non-profit, registered both here and in New York State, USA, with the aggressive mission to "make security make sense." Mostly that means fighting FUD and improving critical thinking skills in the realm of security which includes data and business integrity, development, safety, and trust. Many myths still surround security and only now we're starting to get enough people with open eyes making a difference. Unfortunately there are still far too many parrots out there reciting what they heard about security although it may no longer, if ever, be true or applicable.

Why do we need a security testing methodology? And why open source?

Pete Herzog: Without a security testing methodology, the actual test tends to be all over the place. One tester actually described this once to me as his test being "a mess" without it. The real answer is that a methodology is required to test anything thoroughly. As humans, we take short-cuts. We assume we know an answer or we know what's going on because of past experiences and we cut to the chase because time is money and all that. However, when that happens, we leave many unverified (unanswered) questions and report our assumptions as if they were facts. A good security methodology does not let you do that. A good open source methodology means that many many people don't let you do that. The open source concept actually means that anyone can contribute the ideas for thoroughness and it's not just up to one person, one group, or one authority. While not quite meritocratic as a meritocracy implies, we follow the person with more "wins." In other words, we are democratic as democracy works better for principles and ideas than facts. It is a successful peer review where our reviewers need to show how they got their answers.

How did the project for an Open Source Security Testing Methodology Manual (OSSTMM) start?

Pete Herzog: ISECOM began in January 2001 with the OSSTMM. Actually, the OSSTMM created ISECOM. The truth is really that I wanted to create a plan on how to test security because I didn't think it was being done right and I wanted to improve it. So I searched the net only to find everyone referring to this proprietary methodology they have that's so great. But I couldn't know because I couldn't see it. I was suspicious that it was true because I had seen the reports of some of the companies that said that they had some great proprietary methodology and there was nothing special about what was essentially vulnerability scanner outputs re-dressed as reports. So once I finished something, I posted it to the web and asked the public to give feedback. I had no idea that I was not the only one in need of such a thing. So here we are, five years later and the OSSTMM is at around four million downloads since its inception - with legislation requiring its use in some countries and some government employees and contractors around the world being required to be certified in it just to prove they can really do their jobs. And it's still growing at a fast and shiny pace. We're trying to staff-up to handle this all but that's a problem in itself.

Why did you create a certification process too?

Pete Herzog: The certification process evolved. A need happened which was to do security testing reliably. There are a lot of people with these knowledge certs (the kind that requires knowing or memorizing something) and they didn't seem to get it. They just all made these horrible mistakes when it came to testing. Oh sure, they poked holes and penetrated but were completely incapable of actually really testing security. It was like they tried to light up all the holes in Swiss cheese with a pocket flashlight from 100 meters away. Sure, some holes got exposed but so many more didn't. So we decided to make sure that if we did a certification that it would have to ask the candidates to prove what they know by doing something. So we made the first walk-the-walk security certification of its kind. I'm happy we did it because it adds professionalism and legitimacy to this actually nascent field of security testing. Now it's not sparkly or fancy like certifications on penetration testing or ethical hacking because it's about getting the job done. It's hard work to pass them. It's the difference between rolling up your sleeves to work better and rolling them up to look like you are working. We prefer to help those who really need security and not just look like they have it for compliancy reasons. Then again, we've come so far with only word of mouth so I know we are doing something right.

Some readers might wonder what is OSSTMM concretely about. How does it work? How do you use it?

Pete Herzog: The OSSTMM is concretely about managing and controlling operational security. You can't manage what you can't measure. The OSSTMM gives the power of facts to see what's happening in the operational state of security and make adjustments in the right places as necessary. Some will say risk is a gambler's game and that knowing the operational state can't prevent a break-down (or break-in). That's not true. It is entirely possible to control the state of security and manage which loss controls and which protections you want where but only if you know what's exposed during all points of operations and from all vectors.

Using the OSSTMM is simple. You have a choice. You can get your audits or pen tests as you always have and just now fill in the OSSTMM Audit Report. While it may not be a thorough OSSTMM audit, it will show plainly what has and has not been tested and how. This is a good start to fill in the gaps. The second way is to read and apply the OSSTMM or hire certified OSSTMM auditors (those with their OPSA and OPST). As all OSSTMM audits need to accompany the audit report, you can't go wrong because you will always know exactly what you've gotten.

After five years you are going to release a new major version of the OSSTM Manual. What will we find inside version 3.0?

Pete Herzog: The biggest change is the methodology itself which has evolved from its "steps to security tests" premise which we started with to what is now a true methodology that provides scientific tests and factual measurements for operational security. OSSTMM 3.0 is very advanced in this regard with over 3 years of work into a factual metrics process and concentrated results that qualify as a type of up-time in regards to security. At first it was difficult for people to get used to because they see 99% as their concentrated metric and they get happy. But if you show them it's an uptime and in a year 1% downtime is actually 87.6 hours of downtime due to security problems then they are not so happy. Time is something we can more easily put a financial value on. However, as I said, that 99% does not strictly imply that much downtime as it's actually a metric concentrated (actually hashed) from about 9 different values, each a factual result of a security test. And we can use that 99% to compare security operations from day to day, quarterly, with other test reports even from other testers, and we can combine test results from different test types, channels, scopes, and vectors throughout an organization.

But probably the biggest change we made is to make the OSSTMM accessible to more people through "skins." First we standardized the glossary with ISM3 which itself has been designed to be very compatible and superior to the various, popular, security management standards. We made sure the tests were compatible with how things were done in OSSTMM 2.11 and improvements or necessary changes were logical transitions. Then we made an OSSTMM Audit report which focuses on not just what was tested but what was not and why. This allowed even something as simple as a vulnerability scan to qualify as an OSSTMM audit if it was provided with the OSSTMM Audit report. Then it becomes clear very fast what the metric is actually based on. The skins then evolved from the audit report. People wanted ways to apply the OSSTMM for specific compliance tests like SOX or HIPAA, to complement their ISO270001 audit, for specific industries like banking, or specific technologies like RFID testing. Skins are basically the OSSTMM reduced only to that which needs to be tested for that area and a partially filled in audit report with the specific, metric calculation information included.

Ultimately, what we want is that it's not just a methodology that only a seasoned IT veteran can apply to actually improve real security. We want one that even a typical military soldier can use to assure the physical security of a base or encampment with a metric that shows the weak or redundant security, as well as help field officers report the levels to their commanders to more closely manage day-to-day security operations even from a far-away war room. So you can see what a typical CEO, CIO, CISO, or CFO could do with that kind of fine-grained, security operations information. It would mean hands-on security management capabilities even for those with no formal security background.

Do you plan to develop any tool for OSSTMM users?

Pete Herzog: A project that has been running for a while, yet soon to be released, is a live LINUX distro to support our Hacker Highschool, Security Awareness for Teens project. The distro is based on Gentoo and will include everything needed to run all lessons. Dreamlab Technologies AG, the manager of the distro and a provider of ISECOM certification trainings, is also expanding it to support the OPST and OPSA certification classes.

We also have the Consensus project which is in cooperative development with La Salle University in Barcelona. It is a collection of systems designed to automate an OSSTMM test from multiple vectors like internal network, DMZ, internet-facing systems, and calculate the appropriate metrics. It sends all test data from all vectors to a central system which correlates the data with parsed logs for accurate metrics. Right now it's only a learning tool written in modules to see how much can be tested and how well we can use rules for correlation or even AI techniques. Finally, there are the OSSTMM skins if you count those specific checklists as tools.

Really there's so many more projects and tools in various stages of development that need project managers and support. We've come very far with what we have but more help is appreciated.

Would you like to provide more details?

Pete Herzog: ISECOM has been tapped by the OpenTC project to create the open methodology for testing and assuring integrity in systems using the Trusted Platform Module (TPM). We will use the community effort to get people to take control of the TPM on their motherboards through the creation of tests and processes governing usage, maintenance, application, and integrity. OpenTC is a big project funded by the EU and includes 23 universities and companies like AMD, Infineon, HP, IBM, and SUSE. As much as we all have a love-hate relationship with DRM, with most end-user's feelings being toward hate, we need to take control of that little chip on the motherboard. For example, there are no decent tools out there now that can tell us anything about the chip on our running computers and even less on telling us what's there or how to delete it. It's important that open collaboration steps up to do something about this and even more important if we have the approval of the big corporate suppliers through the OpenTC consortium.

Another area of interest is the vulnerability scanner. We're beginning a project to design an open source scanner that scales to large numbers while maintaining accuracy. We have some bright minds working on this already and we'd like to take this to the next level by not following the way things have been done. What we found is that in security testing, if we rely too much on precedent then we'll never be prepared for the unexpected.

Could you add some technical details about the vulnerability scanner?

Pete Herzog: The vulnerability scanner will first and foremost not be a vulnerability scanner per se as we know it. We are looking to make a security auditor that calculates the security metrics as based on OSSTMM 3.0.

Most people are already asking how it will compare to Nessus. It is not expected to compete on the same level of Nessus nor replace it. I mean let's face it, for what it's designed for, Nessus is really the cat's pajamas. We just need a different approach for our tests.

First, we will deliver the methodology - the framework for it. This will allow it to be initially drafted with existing free and open source tools while applying the rule base and open methodology. We may provide a proof of concept with it to aide in improvement. Meanwhile, we do have already current development work being done on the scanner by three separate groups, each with a different goal. If any one of them decides to move towards being the scanner for this project then great! If not, with the PoC and methodology in place, we should be able to leverage development somewhere fairly easily.

Don't expect a Beta until after the summer. But OPST students may see versions of it in class as early as May.