VMware: VMware vSphere Blog: vSphere HA isolation response… which to use when?

A while back I wrote this article about a split brain scenario<http://blogs.vmware.com/vsphere/2012/05/ha-split-brain-which-vm-prevails.html> with vSphere HA. Although we have multiple techniques to mitigate these scenarios it is always better to prevent. I had already blogged about this before but I figured it wouldn't hurt to get this out again and elaborate on it a bit more.

First some basics…

What is an "Isolation Response"?

The isolation response refers to the action that vSphere HA takes when the heartbeat network is isolated. The heartbeat network is usually the management network of an ESXi host. When a host does not receive any heartbeats it will trigger the response after an X number of seconds. So when exactly? Well that depends if the host is a slave or a master. This is the timeline:

Isolation of a slave

* T0 – Isolation of the host (slave)
* T10s – Slave enters "election state"
* T25s – Slave elects itself as master
* T25s – Slave pings "isolation addresses"
* T30s – Slave declares itself isolated and "triggers" isolation response

Isolation of a master

* T0 – Isolation of the host (master)
* T0 – Master pings "isolation addresses"
* T5s – Master declares itself isolated and "triggers" isolation response

What are my options?

Today there are three options for the isolation response. The responses is what the host will do for the virtual machines running on that host when it has validated it is isolated.

1. Power off – When a network isolation occurs all VMs are powered off. It is a hard stop.
2. Shut down – When a network isolation occurs all VMs running on that host are shut down via VMware Tools. If this is not successful within 5 minutes a "power off" will be executed.
3. Leave powered on – When a network isolation occurs on the host the state of the VMs remains unchanged.

Now that we know what the options are. Which one should you use? Well this depends on your environment. Are you using iSCSI/NAS? Do you have a converged network infrastructure? We've put the most common scenarios in a table.

Likelihood that host will retain access to VM datastores Likelihood that host will retain access to VM network Recommended Isolation policy Explanation Likely Likely Leave Powered On VM is running fine so why power it off? Likely Unlikely Either Leave Powered On or Shutdown Choose shutdown to allow HA to restart VMs on hosts that are not isolated and hence are likely to have access to storage Unlikely Likely Power Off Use Power Off to avoid having two instances of the same VM on the VM network Unlikely Unlikely Leave Powered On or Power Off Leave Powered on if the VM can recover from the network/datastore outage if it is not restarted because of the isolation, and Power Off if it likely can't.

But why is it important…. Well just imagine you pick "leave powered on" and you have a converged network environment and are using iSCSI storage, chances are fairly big that when the host management network is isolated… so is the virtual machine network and the storage for your virtual machine. In that case, having the virtual machine restarted will reduce the amount of "downtime" from an "application / service" perspective.

I hope this helps making the right decision for the vSphere HA isolation response. Although it is just a small part of what vSphere HA does, it is important to understand the impact a wrong decision can have.

________________________________

Original Page: http://blogs.vmware.com/vsphere/2012/07/vsphere-ha-isolation-response-which-to-use-when.html