Novell News

IT product reviewers have an interesting job. They test drive the newest, most in-demand technology to help their readers make intelligent purchasing decisions. For the readers and vendors of IT magazines, there’s a certain amount of mystery about reviews and the judging criteria.

One of the industry’s best-known and most respected reviewers, Tom Henderson of ExtremeLabs Inc. in Indianapolis, connected with the Cloudchasers Internet radio program this past week to shed some light on the life of a reviewer.

Tom is best known for the reviews his team publishes in NetworkWorld. Recently, the magazine reviewed a number of products companies can use to create and manage private clouds. Novell Cloud Manager was one of the products reviewed.

This Dec. 20, 2010, review was the fourth in a series of product reviews NetworkWorld has done on cloud computing. The others include an array of management APIs the public cloud operators deploy for private use, developer-focused applications that control the public cloud, and products used for public cloud storage.

When testing these products, what is Henderson’s team looking for?

“We’re trying to find out what features the product supports, how the product is constructed procedurally, how is it administered, what it looks like from a user perspective (and) what security components are involved or adhered to in the process,” he said. “And then a more subjective look at, how did we like it. Did we have to drink too much coffee? Is there a lot of soap opera? Are the docs in a language that look like English but we’re not sure? That’s how we do it.”

He said there are unique challenges in the testing of cloud tools.

“Because we’re reviewers of new stuff — bleeding edge — we sometimes find the products don’t work,” he said. “We often have to go and deal with bug fixes – we call it the ‘dot zero syndrome.’ Sometimes we help vendors discover new and interesting quirks about their software.”

Just picking a representative sample of products in the private cloud management arena was difficult, Henderson said, because the products in the market are so different. He said the team tried to pick from three traditional classes of applications to support:

1. products that manage line-of-business applications such as email, office automation tools (“the bread and butter of what servers have done eons”)
2. commonly reused but non-persistent applications focused toward traditional job control (“things you do once a month”)
3. tools that manage “one-shot-focused jobs” such as video and media rendering or mining Census data for intelligence.

“The idea is to show several different kinds of approaches to this private cloud management, newly emerged category, so readers have an idea of what they taste like in these different areas,” Henderson said. Asked if it was like comparing apples to oranges, he said it was more complicated.

“More like tangerines vs. mandarin oranges vs. tangelo,” he laughed.

Here is the link to the Cloudchasers podcast featuring Henderson and John Stetic of Novell.