When VMs Go Rogue

As small businesses turn to server virtualization as a way to consolidate their data centers and become more energy and cost efficient, they’re realizing that there can be too much of a good thing. Some IT shops are looking for ways to keep their virtualization efforts under control and prevent so-called virtual machine sprawl from becoming unmanageable.

The best way to control the number of virtual servers in the data center is to “lay down the law” to users and enforce it, says Sean Smith, director of network operations at website development, interactive marketing and web hosting company simpleview. In Smith’s case, the law consists of strict policies defining acceptable reasons for creating virtual machines and designating who is authorized to create them.

“We haven’t had the virtual server sprawl experience yet because we have processes in place to prevent it,” Smith says.

Users have to specifically request a new virtual server at simpleview. The approval process starts with Smith and goes on to the chief technology officer. The company sets expectations early about use of the virtual environment, and it makes few exceptions, Smith says. The result is that people generally make smart requests, he says.

Companies that have deployed server virtualization in their data centers report significant savings in space, energy and administrative costs. But their IT staffs also agree that there is a downside to the ease with which virtual machines can be created. Unbridled proliferation of VMs by enthusiastic users may result in management migraines, misdirected computing resources and security nightmares.

Although there are plenty of products available from major virtualization manufacturers to help streamline management, control configuration and automate processes, the most effective check on virtual server sprawl lies in well-conceived policies rather than technology, say IT managers.

Simpleview, based in Tucson, Ariz., made the move to server virtualization about two years ago because rapidly multiplying physical servers were gobbling up rack space and electricity in the data center. Using VMware’s ESX virtualization platform, simpleview reduced its physical servers from 17 to three, which together host more than 22 virtual servers and are connected via iSCSI to a storage area network. Even though the SAN delivers 12 terabytes of storage, rationing that space is one of the drivers of the policies that govern the creation of virtual servers, Smith says.

“Twelve terabytes sounds like a lot of space, but it disappears pretty quickly,” he explains. “We’re using virtualization for our core functions, and we don’t have a need for a lot of specialty servers. Developers occasionally ask if they can take one of the unused physical boxes and build some virtual machines, and I say, ‘No, you can’t build something just because you read an article and it sounds cool.’”

Define Policies

Small and midsize companies often underestimate the need to establish defined policies and procedures for the virtual environment, says Dominic Foster, CTO of MaximumASP, a web hosting and applications service provider in Louisville, Ky.

“Many smaller companies don’t even think about setting policies when they start to virtualize,” Foster says. “They think, this is a panacea, it’s going to fix all of our problems. You can definitely create a lot of new problems that way.”

Sprawl is not new with virtualization, says Foster. Plenty of servers are shoved into closets and under desks without much thought. But it’s much easier to add virtual servers, especially if there are too many people with administrative privileges. They can spin up a VM in a few minutes, and then it’s out there in a rogue state, Foster says.

MaximumASP offers a range of service models to its customers, from shared hosting to dedicated virtual and physical servers. The company started exploring virtualization soon after it was founded in 2000, when it quickly became apparent that its data center was running short on space and power, Foster says. After trying out several solutions, MaximumASP settled on the Microsoft Hyper-V virtualization platform and runs it on Windows Server 2008 Release 2, Enterprise.

Although good policies are crucial to managing the virtual environment and controlling virtual server sprawl, they should be applied in concert with the available tools, Foster says. MaximumASP uses the full System Center Virtual Machine suite: Configuration Manager to patch VMs; Operations Manager to monitor them; Data Protection Manager for backup; and Virtual Machine Manager for orchestrating the entire virtual environment.

“Virtual Machine Manager lets you manage the host machine so you can manage the virtual machines on top of it,” Foster says. “You can also run provisioning scripts on top of it, which automate the rules and go a long way toward eliminating rogue VMs and sprawl.”

Managing VMs as though they were physical machines is the best way to prevent sprawl in a virtual environment, says Nicholas Tang, vice president of technical operations at Interactive One in New York.

“We use a lot of centralized configuration management tools,” Tang says. “We typically don’t just build virtual machines out randomly. We have automated processes for building, deploying and configuring servers. Once they’re out there, we have automated processes for inventorying servers. We also have a limited number of physical servers that we rope off and allow to run as virtual hosts, so that keeps a check on VM proliferation.”

Interactive One, which runs BlackPlanet.com (one of the largest social networking sites) four uniquely programmed websites and more than 50 other online media properties for parent company Radio One, uses Oracle Enterprise applications as the central software system in its data center. The Oracle VM virtualization platform was a prime selling point for the suite, Tang says.

Since the company began using server virtualization, the number of physical servers in its data center has dropped from 450 to 150, he says. But while consolidation of resources and ease of deployment and management have made virtualization an important tool for Interactive One, it’s one that is used strategically where the fit is right for the task, Tang says.

“If you’re trying to get every last bit of performance out of every server box, that doesn’t necessarily mean putting multiple VMs on all of them — virtualization is not magic fairy dust,” he says.

Tang says there are lots of cases where an app can max out the performance of a box: a database or a heavily hit web server or file server, for instance. “It may look like you’ve got excess capacity for a VM, but it’s not worth the RAM or the management resources you’d lose,” he says.

Early on, Interactive One’s heavy reliance on batch processing spurred the deployment of server virtualization. Virtual servers are well suited to applications that work in the background as utilities, such as mail servers and boxes dedicated to batch processing, Tang says.

“The secret to containing virtual server sprawl is to make sure you need virtualization for the application you’re considering, and then use the appropriate tools to configure the VMs and manage them and the host boxes,” he says.

Managing Resources

EmpireCLS Worldwide Chauffeured Services in Norwood, N.J., is already realizing benefits from its server virtualization initiative on Microsoft Hyper-V, says CIO Alan Bourassa.

The company, which offers chauffeur services in more than 650 cities worldwide, expects a 50 percent reduction in the physical servers it uses; a 33 percent drop in the amount of electricity it consumes; and an environment that is easier to manage, back up and restore than one limited to physical boxes. Balanced against those advantages, preventing virtual server sprawl is a minor challenge, Bourassa says.

“It’s all about the IT shop and how it manages its resources,” he says. “You get virtualization sprawl when you configure the operation to allow users to, at will, spin up new machines. That’s a feature we don’t enable. If you were using physical servers, that’s certainly not the way you would do it.”

Like other IT managers who stress the importance of policies to manage virtual server sprawl, Bourassa does not discount the value of available tools. Microsoft System Center Virtual Machine Manager offers a GUI console that lets the IT staff monitor and control the entire virtual environment, including any potentially rogue VMs, from a central point, he says.

With the right policies and tools in place, virtual environments can be easier to manage than physical servers. The best practices for preventing server sprawl are consistent with effective IT management: Make sure the technology is the right fit for the job and continue to monitor its performance.

“Virtualization provides so many possibilities that some of its uses can be out of place in a given IT environment,” says Foster of MaximumASP. “Create rule sets of your own, tailored to your environment, and then automate as many of them as you can. And, of course, remember the old rule: You can’t manage what you don’t monitor.”