IBM Research       


IBM Research - Turtles all the way down

IBM researchers are using nested virtualization to turn a single virtual machine into many machines, each with the potential to have its own unique environment.

turtle What is nested virtualization and who needs it?

Classical virtualization takes a physical computer and turns it into multiple logical, or virtual, computers. Each virtual machine can then interact independently, run its own operating environment and basically behave like a separate physical resource. Hypervisor software is the secret sauce that makes virtualization possible by sitting in between the hardware and the operating system. It manages how the operating system and applications access the hardware.

IBM researchers found an efficient way to take one x86 hypervisor and run other hypervisors on top of it. For virtualization, this means that a virtual machine can be "turned into" many machines, each with the potential to have its own unique environment, configuration, operating system, or security measures -- which can in turn each be divided into more logical computers, and so on. With this breakthrough, x86 processors can now run multiple ‘hypervisors’ stacked, in parallel, and of different types.

This nested virtualization using one hypervisor on top of another is reminiscent of a tale popularized by Stephen Hawking. A little old lady argued with a lecturing scientist and insisted that the world is really a flat plate supported on the back of a giant tortoise. When the scientist asked what the tortoise is standing on, the woman answered sharply, "But it's turtles all the way down!" Inspired by this vision, the researchers named their solution the Turtles Project: Design and Implementation of Nested Virtualization.

They said it wasn’t possible

“In systems research, sometimes the boundary between research and engineering gets a little fuzzy,” says Muli Ben-Yehuda, research scientist and one of the leaders of the Turtles team. “We built something real that works, but it’s also something that was thought to be impossible. We couldn’t resist the challenge – it’s got research written all over it.“

Although virtualization was first introduced in the 1960s by IBM to boost utilization of large mainframe systems, the new distributed environments required for cloud computing have brought virtualization solutions back into the limelight.

The researchers are already seeing the first sparks of their work’s influence and its impact on both industry and academia. The work already has spawned numerous citations and computer science classes in several universities have included the Turtles paper in their course materials.

What can you do with nested virtualization? More than you think

“With nested virtualization for x86, it is possible to envision embedded hypervisors for Intel-based servers, much as IBM does for System P and System Z; the user can run their hypervisor of choice, on top of the embedded hypervisor,” noted Abel Gordon, research scientist and another one of the leaders for this breakthrough. Nested virtualization can be also used to enhance cloud offerings with new exciting capabilities:

  • A person who has their business running on a virtual machine in some provider’s cloud environment, can now run their own hypervisor on top of the virtual machine. By adding as many virtual machines as they like, they can in essence create their own cloud.
  • By using one machine to build a small cloud with a number of hypervisors, we can simulate how code and software will work on a bigger cloud with many machines.
  • When data center prepare for moving to the cloud, nested virtualization can be used to replicate the customer’s environment by using nested virtualization to represent all the different machines and configurations at the original site.

Where to now?

Gordon explained that the researchers began their work using the Linux Kernel-based Virtual Machine (KVM) core from Red Hat. Because of the lack of architectural support for nested virtualization, an x86 guest hypervisor cannot use the hardware virtualization support directly to run its own virtual machine. The research team developed software extensions for KVM to multiplex multiple levels of virtualization (multiple hypervisors) on the single level of architectural support, thereby making efficient nested virtualization possible.

After a year of intense and ongoing iterations, handled by fellow researcher Nadav Har’el with the open source community, the IBM solution is now part of official Linux distributions. The team will continue to maintain the code, fix errors, and send extensions. “We’re also looking forward to seeing this go out to IBM customers and service offerings,” continued Gordon. “It opens a lot of exciting opportunities.”

Read more about nested virtualization at the IBM Research - Haifa Lab.

This article first appeared on the IBM intranet. It was written by Chani Sacharen.

Last updated on September 1, 2011