Virtualization: A brief architecture overview of VMware ESX, XEN and MS Viridian

Massimo Re Ferrè has written a nice overview article on hypervisor architecture:

It is my feeling that there has been a bit of confusion lately around how hypervisors are being positioned by the various vendors. I am specifically referring to the three major technologies that seem to be the most relevant strategically going forward:

• VMware ESX
• Microsoft Viridian
• Xen

VMware ESX is the VMware flagship hypervisor product: it's the basis for the Virtual Infrastructure version 3 framework.

MS Viridian is the next generation hypervisor that Microsoft is going to use in the Longhorn time frame and that is currently being developed. It's basically the successor of Microsoft Virtual Server.

Xen is an opensource hypervisor that is being integrated by a number of players which include RedHat, Suse, XenSource and Virtual Iron.

Read the article here

VMware: VirtualCenter out of sync

I entered the office and fired up my trusty VIClient to connect to our design environment, just to discover all machines were shutdown. Suddenly the phone started ringing. All production VMs were shutdown as well? What happened?

It turned out to be a massive (physical) switch failure: during the weekend, all switches were failing in cascade due to some switch intervention (oeps). Result: all NICs in all ESX hosts lost their connection. So, every ESX host had noticed it couldn't ping it's default gateway and started shutting down ("releasing") all VMs (the default HA behavior). But...there were no other ESX hosts to pick up the host failure :)

After booting the VMs, we noticed something strange: VirtualCenter was completely out of sync with what was running on the ESX hosts themselves and we couldn't VMotion/Edit Settings/.. our VMs. Some VMs appeared to be running on a host when they were not! Really strange, strange stuff.

To get your VirtualCenter back in sync (and if rebooting the VC Service doesn't help), you must restart the vpxa (VirtualCenter) agent on every ESX host by issuing the following command:

/etc/rc.d/init.d/vmware-vpxa restart

VMware: The dreadful sticky snapshot

When working with VMware ESX’s snapshot technology and/or VCB for a while, you will notice that sometimes a snapshot doesn’t get committed correctly. This occurs when for instance an ESX host crashes or hangs during a VCB snapshot.

The problem here is that a redo file (or delta, snapshot file) is created, but never removed. When looking at the snapshot manager, no snapshots appear to be available. Snapshots are devastating for your datastore performance, so we must remove them as quickly as possible! In the worst case, the redo file and the vmware-x.log files will eat up all the remaining disk space on the infected datastore and a message will be displayed (AAAaaaaaaaaaahh):

You will notice a number of <originaldiskname>-00000X.vmdk and <originaldiskname>-00000X-delta.vmdk files appear when browsing the datastore. When you look at your Virtual Machine disk properties (via Edit Settings) you will notice that your disk name will have changed from <originaldisk.vmdk> to <originaldisk>-00000X.vmdk, meaning a snapshot version is being used.

You can use 2 tricks to recover from this scenario:

• Execute vmware-cmd /vmfs/volumes/<datastorename>/<vmname>/<vmname>.vmx removesnapshots on the Service Console. However, most of the time this will not work and you will have to revert to the method below.
• Manually create a snapshot in the VI Client and remove it after it has been created. This will remove ALL existing snapshots and revert back to your <originaldisk>.vmdk. Great huh?

VMware: VMTSPatchmanager for patching

Massimiliano Daneri of vmts.net (well known for his backup scripts) has created a useful tool for patching your ESX: VMTSPatchmanager.

Features:

• Automatic Patch downloading from VMware Web site
  Virtual Center Integration
• Support for Background Intelligent Transfer Service (Microsoft BITS)
• Password saved inside the Microsoft Protected Storage
• SSH Connection (with root or other users)
• Implement a tiny Web server to deliver the patch
• Multi Thread implementation
• Configuration Wizard

Download it here

Microsoft: Terminal Services Session Broker

Michel Roth over at thincomputing.net has written a nice (p)review on the new load balancing options for terminal servers in Server 2008. Read his article here