vSphere ESX4: Hot Add CPUs for Linux Guests

Sometime ago within my post entitled vSphere ESX4: Hot Add Memory for Linux Guests, I promised to blog about hot add CPU support to the same VM.  As you may have guessed, I didn’t get around to writing it, and have subsequently found the area well covered elsewhere.

wila has knocked together a very clear procedure here http://communities.vmware.com/docs/DOC-10493 which kind of makes my post unnecessary.

My earlier post has been referenced as a resource by lamw for his helpful guide here: http://communities.vmware.com/docs/DOC-10492

Add to FacebookAdd to NewsvineAdd to DiggAdd to Del.icio.usAdd to StumbleuponAdd to RedditAdd to BlinklistAdd to Ma.gnoliaAdd to TechnoratiAdd to Furl


vSphere ESX4: Hot Add Memory for Linux Guests

I was asked recently if I could hot-add some RAM to a client’s virtual machine over at virtualDCS. Most of the information I’d found online related to Windows Server versions, but I needed to hot add resources to a Linux VM. VMware.com was lacking in detail about the hot-add compatibility with client operating systems, so I realised I’d better lab it up and see how it works for myself.

The first problem I had, was that the virtual machine I’d cloned from my clients live VM, was originally built using ESX3.5. Hence, it was VM version 4, and hot add hardware is not supported unless the VM hardware is upgraded to version 7. In order to enable hot-add features, I had to first upgrade VMware Tools, and then shut down the VM again to upgrade the virtual hardware to version 7.

Once this had been done, I made sure the VM General Options (VM > Edit Settings > Options > General Options) was set to the correct OS type. This important, as the interface will only display the Memory/CPU Hotplug options for supported OSes. In my case I was running CentOS 5.3 x86_64, so selected Other Linux 2.6.

General Options

Next I enabled the Hot Add CPU and Memory as below, but was unable to check the radio button for Hot Remove CPU, which is interesting in relation to what I found when playing with Hot Add CPUs (discussed in an upcoming post).


I found that the CentOS build I was using (2.6.18-128.el5) recognises hot added memory automatically. A colleague (thanks Stu) recommended I read the Linux Hotplug Memory docs which made the rest fairly obvious.

My VM was running with 512MB RAM, so I added some more via the vCenter console, so my VM now had 1GB RAM allocated to it. (BTW: even though vCenter appears to let you do this for the 32bit guest version, it doesn’t actually work. The task is reported as successful, but when you check the VM properties again, you’ll see the RAM was not added.)

When memory is hotplugged, the kernel recognizes new memory, makes new memory management tables, and makes sysfs files for new memory’s operation.
If firmware supports notification of connection of new memory to OS, this phase is triggered automatically. ACPI can notify this event. If not, “probe” operation by system administration is used instead.

Now comes the interesting part. Within


there are a number of folders all named ‘memoryX’ where X represents a unique ‘section’ of memory. How big each section is, and hence how many folders you have is dependent on your environment, but you can check the file


to view the size of sections in bytes. Basically, the whole memory has been divided up into equal sized chunks as per the SPARSEMEM memory model.

In each section’s folder there is a file called ‘state’, and in each file is one of two words; online or offline.
Locate the memoryX folder(s) which account for the hot added memory by working out the section sizes above, or (like me), just check the contents of the state files:

#cat /sys/devices/system/memory/memoryX/state

Once you locate the offline sections, you can bring them online as follows:

#echo online > /sys/devices/system/memory/memoryX/state

Validate the memory change is seen, using:


That’s it! Quite simple really.

UPDATE: I noticed that William Lam (lamw on the VMware communities) created a nice script to automate the discovery and online process.  It’s very neat and can be downloaded here:

# William Lam
# http://engineering.ucsb.edu/~duonglt/vmware/
# hot-add memory to LINUX system using vSphere ESX(i) 4.0
# 08/09/2009

if [ "$UID" -ne "0" ]
 echo -e "You must be root to run this script.\nYou can 'sudo' to get root access"
 exit 1

for MEMORY in $(ls /sys/devices/system/memory/ | grep memory)
 echo "Found sparsemem: \"${SPARSEMEM_DIR}\" ..."
 STATE=$(cat "${SPARSEMEM_STATE_FILE}" | grep -i online)
 if [ "${STATE}" == "online" ]; then
 echo -e "\t${MEMORY} already online"
 echo -e "\t${MEMORY} is new memory, onlining memory ..."
 echo online > "${SPARSEMEM_STATE_FILE}"

Add to FacebookAdd to NewsvineAdd to DiggAdd to Del.icio.usAdd to StumbleuponAdd to RedditAdd to BlinklistAdd to Ma.gnoliaAdd to TechnoratiAdd to Furl

Free: Citrix Essentials for Hyper-V

Fascinating movement in the virtualisation space currently. With Hyper-V R2, MS will be providing a FREE virtualisation platform which includes Live Migration, and High Availability as standard. This supports up to 1TB RAM per host, up to 8 CPU’s per host, and up to 16 nodes per failover cluster.

Add to that, the new Windows 7 management interface that means you no longer have to use SCVMM or Windows Server Management tools, and you have a bargain.

Now it seems that Citrix are getting into the mix, by offering their Citrix Essentials for Hyper-V for FREE too.

Citrix Essentials for Hyper-V also includes their StorageLink technology, enabling storage provisioning and management for iSCSI and FC SAN environments.

A lack of GUI admin tools can be a disadvantage for small teams wishing to deloy Hyper-V, so utilities like this one from Citrix should really help these teams with their implementations.

Add to FacebookAdd to NewsvineAdd to DiggAdd to Del.icio.usAdd to StumbleuponAdd to RedditAdd to BlinklistAdd to Ma.gnoliaAdd to TechnoratiAdd to Furl

and now, the Google Chrome OS…

After the raging success of Google’s Android phone OS (ok, NOT), Google are now leaping into everyday computing OS with their Google Chrome Operating System.

It will be released initially for netbooks, but won’t be open source <gasp>, until later this year <exhale>.

One comment in the above article that worries me is:

“And as we did for the Google Chrome browser, we are going back to the basics and completely redesigning the underlying security architecture of the OS so that users don’t have to deal with viruses, malware and security updates. It should just work.”

I mean, it’s not as if Windows had viruses the first day it was released.  All I’ll say is ‘if you build it, they will come’.

It’s nice to have competition in the desktop space, but until there is a narrower Linux distro base, all this variety will help MS dominate.  Apple’s Linux disto works well primarily due to aesthetics of their kit and serious branding.  It’ll be interesting to see how Google take theirs to market.

Add to FacebookAdd to NewsvineAdd to DiggAdd to Del.icio.usAdd to StumbleuponAdd to RedditAdd to BlinklistAdd to Ma.gnoliaAdd to TechnoratiAdd to Furl

VMware vShield – was it worth it?

I just spent a couple of hours happily deploying VMware vShield Zones, less happily pouring over the manuals, and then unhappily thinking I’d wasted my time.

I think our ESX platform is fairly typical. We have multiple ESX servers, running guest VM’s for multiple customers (or departments), many of which are tagged to isolated vLans, and most of which ultimately communicate to the outside world via our firewall clusters. To achieve security in this scenario means understanding your vlans, dropping the right vNic on the right VM, and managing a typical firewall appliance (Cisco in my environment).

VMware vShield Zones have been introduced (actually bought from Blue Lane Technologies) supposedly to simplify the network security by implementing a firewall within your ESX farm. Sounds cool, right? It would be too, if it was done right.

I won’t go into the detail of how it works, and how to configure it, as you can read up on that by following the links on Rodos‘ blog.
There are loads of gotchas, and strange concepts at first, but they’re all well documented in the manual. The install process was flawless too, so what’s not to like?


  • It requires a vShield agent VM per vSwitch with a physical NIC attached. That means lots of additional VM’s for us.
  • It does not offer anywhere near enough reporting detail. No real time bandwidth monitors, just per hour statistics.
  • It does not offer any bandwidth controls like rate limiting or QoS.

On the contrary, as I doubt anybody will be throwing out their perimeter firewalls just yet, vShield adds a further layer to manage. Perhaps I’m missing something.

Add to FacebookAdd to NewsvineAdd to DiggAdd to Del.icio.usAdd to StumbleuponAdd to RedditAdd to BlinklistAdd to Ma.gnoliaAdd to TechnoratiAdd to Furl


Oops how embarrassing!

I often stumble upon an interesting blog or website, but am usually reluctant to add it to my favourites.  My favourites is full of clutter from broken links, retired sites, and urls that are quicker just to type in.

What I need is a web service that provides a list of favourite sites, which saves me synchronising my favourites within my Mesh, and also allows me to share links I think are interesting, but not worth blogging about.

Enter Ma.gnolia.com (I have no idea why they write it like that).  Ma.gnolia is, well, let them tell you:

At Ma.gnolia, members save websites as bookmarks, just like in their browser. Except with a twist: they also “tag” them, assigning labels that make them easy to find again. So when you search for something, you use words that people choose and look only at websites that people think are worth saving. Suddenly you have access to a human-organized bookmark collection that numbers in the millions, but is as easy to use as a search engine.

With Ma.gnolia, that’s really all the work you have to do. Finding by tags makes organizing bookmarks a thing of the past. Since it’s a website, your Ma.gnolia bookmark collection can be reached by you and your friends from anywhere, any time. And don’t worry about web pages disappearing from your searches or even the web, as we make a saved copy of each page you bookmark where websites allow us to.

All very interesting, but one of the main reasons to use the service is so that you always have access to your favourites.  Unless they lose them of course.

A couple of days ago, that’s exactly what they did.  And they can’t get them back.  Here’s what they have to say (link):

Dear Ma.gnolia Community Member or Visitor,

Early on the West-coast morning of Friday, January 30th, Ma.gnolia experienced every web service’s worst nightmare: data corruption and loss. For Ma.gnolia, this means that the service is offline and members’ bookmarks are unavailable, both through the website itself and the API. As I evaluate recovery options, I can’t provide a certain timeline or prognosis as to to when or to what degree Ma.gnolia or your bookmarks will return; only that this process will take days, not hours.

I will of course keep you appraised here and in our Twitter account.

Most importantly, I apologize to all of you who have made Ma.gnolia a home for your bookmarks and community. I know that many of you rely on Ma.gnolia in your day to day work and play to safely host you bookmarks, keeping them available around the clock, and that this is a difficult disruption.


Oh dear.

I’m especially surprised by the “as I evaluate recovery options” comment.  Surely every business understands their recovery options.  Don’t they?

When online presence is crucial (i.e. your main business function), as it is with web service providers, a fast recovery plan should have been in place.  Replication of the data to a second location, with regular snapshots to protect against data corruption, is such an inexpensive protection strategy nowadays.  Add to that the ease with which service providers can test the recoverability, this failure is a true schoolboy error.

The lesson to be learned for the rest of us, is to take DR plan into your own hands.  Store multiple copies of the data you want to keep.  Fortunately a very helpful blogger, Hutch Carpenter, posted a great idea to make this a simple process.  Store your bookmarks at Diigo, and let Diigo copy them to Del.icio.us.  See his site for a step by step guide.

Add to FacebookAdd to NewsvineAdd to DiggAdd to Del.icio.usAdd to StumbleuponAdd to RedditAdd to BlinklistAdd to Ma.gnoliaAdd to TechnoratiAdd to Furl

VMware and iSCSI – explained

A colleague alerted me to a great post regarding iSCSI performance with specific reference to VMware ESX hosts.

I know many organisations operating VMware farms with iSCSI storage systems, and I expect many will fall foul of some of these excellent gotchas.  The most important of which is that you should really have multiple iSCSI targets if you want to maximise your performance.  Hence, make sure your iSCSI storage hardware supports presentation of LUN’s as individual targets.

Add to FacebookAdd to NewsvineAdd to DiggAdd to Del.icio.usAdd to StumbleuponAdd to RedditAdd to BlinklistAdd to Ma.gnoliaAdd to TechnoratiAdd to Furl