Vulnerability management tools in a nutshell

Capsule reviews of Critical Watch, eEye, Lumension, Qualys, McAfee and SAINT.

We tested six market-leading products and evaluated each for their vulnerability scanner results, reporting features, product manageability, workflow tools, and interoperability with other enterprise products. Here are capsule reviews of each product.

We tested FusionVM in its software-as-a-service configuration, giving us a portal-based vulnerability analyzer with off-site and on-site scanner capability. (Critical Watch offers other packagings which are entirely on-site if needed.)

We tested six market-leading products and evaluated each for their vulnerability scanner results, reporting features, product manageability, workflow tools, and interoperability with other enterprise products. Here are capsule reviews of each product.

Critical Watch FusionVM Enterprise

We tested FusionVM in its software-as-a-service configuration, giving us a portal-based vulnerability analyzer with off-site and on-site scanner capability. (Critical Watch offers other packagings which are entirely on-site if needed.)

Do you know where your security holes are?

We found it nice to be able to quickly deploy scanners in virtual machines. Having the flexibility to scan from the inside or the outside also gives additional benefits. With strong feature sets focusing on delegated management and compliance, FusionVM has a clear emphasis on the compliance marketplace. We found the reporting to be a strong feature, and the built-in Web vulnerability testing features will be interesting to anyone who fears bugs in their externally facing Web sites.

FusionVM also has a direct link to TippingPoint IPS products, offering the ability to optimize IPS configurations based on real detected vulnerabilities and systems.

In comparing FusionVM to other vulnerability analyzers, we found much to like and some features that didn't thrill us. The documentation is out-of-date and, in many of the places we went looking, poorly executed. The Web-based GUI, so critical to management of FusionVM, didn't work very well in our environment, with some features such as remediation and workflow blocked almost completely, while other usability flaws got in our way for common operations. If these features can be remedied, FusionVM will be a strong competitor, but there is work to be done at Critical Watch.

eEye Retina CS

Retina CS (Compliance + Security) is a reporting and compliance toolkit and GUI that sits on top of the venerable and well-respected Retina Network Security Scanner. CS is a relatively new product (about a year old) and shows some rough edges, scanner and GUI bugs, and design flaws.

Compliance considerations

But the product is under active development. We saw one upgrade of Retina CS during our test, and one slated to appear after we were done (with a feature that we really wanted, exclusion listing).

Retina CS stood out for its easy-to-define report formats. What we found missing, though, was solid integration between the scanner reporting engine and the database of scan information, requiring not only separate GUIs but even separate authentication systems. The strong set of third-party partners reflects the maturity of the company and its long-time presence in this marketplace. Retina CS is a favorite underdog, with some great ideas and technology that need further refinement and a good dose of bug fixes. When Retina CS does finally become enterprise-ready, it'll be a great tool for network managers who have systems, Web site, and compliance concerns.

Lumension Scan

Lumension Scan is a very nicely constrained product that doesn't try to reach beyond its base functionality and capabilities. Our extensive enterprise feature set was a bit of a challenge for Lumension, and so it didn't rate so well in our scorecard, but this doesn't mean it isn't a fine product — it just doesn't have the bells and whistles we were looking for in this test.