Thursday, December 20, 2012

Set the Scratch Partition from the vSphere Client

If a scratch partition is not set up, you might want to configure one, especially if low memory is a concern. When a scratch partition is not present, vm-support output is stored in a ramdisk.
The directory to use for the scratch partition must exist on the host.


Use the vSphere Client to connect to the host.

Select the host in the Inventory.

In the Configuration tab, select Software.
Select Advanced Settings.
Select ScratchConfig.
The field ScratchConfig.CurrentScratchLocation shows the current location of the scratch partition.

In the field ScratchConfig.ConfiguredScratchLocation, enter a directory path that is unique for this host.

Reboot the host for the changes to take effect.

(copy from vSphere documentation)

For automated scratch partition configuration you can use vCLI, PowerCLI. For details see. VMware KB 1033696.

And here is my PowerCLI script inspired by KB above to set scratch location on all ESXi hosts in particular vSphere clusters.

Wednesday, December 19, 2012

ESXi strange related log entry in /var/log/vmkernel.log

I've just found in /var/log/vmkernel.log lot of following storage errors

2012-12-19T01:34:02.010Z cpu2:4098)NMP: nmp_ThrottleLogForDevice:2318: Cmd 0x93 (0x412401965f00, 5586) to dev "naa.60060e80102d5f500511c97d000000d4" on path "vmhba2:C0:T0:L2" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x96 0x32. Act:NONE
2012-12-19T01:34:02.010Z cpu2:4098)ScsiDeviceIO: 2322: Cmd(0x412401965f00) 0x93, CmdSN 0xc6fd5 from world 5586 to dev "naa.60060e80102d5f500511c97d000000d4" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x96 0x32.

The main part of log entry is "failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x96 0x32"

If I understand correctly
Sense code 0x5 = ILLEGAL REQUEST

What is it? What doe's it mean?

I have ESXi 5.0 build 768111, storage HDS AMS 2300, CISCO UCS blade system, CISCO FC switches.

Update 1:
I've thought more about the root cause ... important detail is that it is happen when storage vMotion or other data migration is happening. So I've a hypotheses that it is related to VAAI. Storage is VAAI enabled and VAAI is supported. However disk block size is different on datastores (we are just in the middle of migration from VMFS-3 to VMFS-5).

So I've to do deeper diagnostic and root cause troubleshooting.

Stay tuned.

Update 2:
Solved, VAAI primitives must be enabled also on HDS Host Masking. For more information check

Friday, December 07, 2012

Storage Queues and Performance

VMware recently published a paper titled Scalable Storage Performance that delivered a wealth of information on storage with respect to the  ESX Server architecture.  This paper contains details about the storage  queues that are a mystery to many of VMware's customers and partners.   I  wanted to start a wiki article on some aspects of this paper that may  be interesting to storage enthusiasts and performance freaks.

Blog post for more information is at

These information are very useful for deep understanding of full storage stack.

Wednesday, December 05, 2012

Best Practices for Faster vSphere SDK Scripts

Source at 
The VMware vSphere API is one of the more powerful vendor SDKs available in the Virtualization Ecosystem.  As adoption of VMware vSphere has grown over the years, so has the size of Virtual Infrastructure environments.  In many larger enterprises, the increasing number of VirtualMachines and HostSystems is driving the architectural requirement to deploy multiple vCenter Servers.
In response, the necessity for automation tooling has grown just as quickly.  Automation to create daily reports, perform bulk operations, and aggregate data from large, distributed Virtual Infrastructure environments is a common requirement for managing the increasing virtual sprawl.
In a Virtual Infrastructure comprised of thousands of objects, even a simple script to list all VirtualMachines and their associated HostSystem and Datastores can result in very slow runtime execution.  Developing automation with the following, simple best practices can take orders of magnitude off your vSphere API tool's runtime.


Monday, December 03, 2012

DELL Active System Manager

DELL Active System is managed by DELL Active System Manager. This is DELL converged infrastructure solution (blade server, networking, storage) to achieve "mainframe of 21st century" with leveraging server virtualization (hypervisors) to have enough flexibility to achieve required infrastructure SLAs.

Configuring a Chassis in Dell Active System Manager

IBM PureFlex

IBM Pure Flex System is probably another next generation computing system leveraging converged infrastructure concept. IBM Flex System Manager manages Pure Flex System. Who can honestly and precisely compare it with HP Virtual Connect, CISCO UCS, and DELL Active System?

Introduction video is available at

Saturday, December 01, 2012

VAAI - VMware API for Array Integration deep dive