My VMware ESXi 4.1 home lab

With my new role at VCE as a Solutions vArchitect and Virtualization evangelist I get to work with a lot of cool stuff. Tech I want to test myself and it makes a lot of sense to have a home lab for testing and professional development. Sure there are lab resources at work and we get to work with all that cool tech but the environment is dynamic. A home lab has other uses besides learning VMware and testing. There are some examples on lifehacker.com Television personal video recorder, Secure Wireless connections when using public WiFi. Each of these can be a VM running in the lab.

I have read a number of articles on home labs and wanted to add my experience with building one. As we all know technology changes quickly and it may be helpful to share what I’m finding success with as of March, 2011. Many blog posts out there cover hardware from 2009 when desktops were using DDR2 memory and Intel Core 2 Duo processors. These systems were limited to 8GB of RAM. This year second generation Intel Core i3, i5 and i7 processors are available with prices on first generation dropping. These processors and P55/H55 chipsets use DDR3 memory which allow for 16GB of RAM (double the amount of a DDR2 memory system). Intel Core i3/i5/i7 system prices are more expensive but worth it for the amount of RAM you can pack in. I also priced AMD processors but the price gap is small when comparing current technology.

I spent under $375 per PC, running bare bones (no case, monitor, keyboard or HDD).  About $450 per PC with case and HDD. Newegg.com, buy.com, tigerdirect.com and Frys seem to offer the best prices on computer parts (please leave comments with other good sites). Prices on PC parts change with the weather depending on weekly sales, mail in rebates and changes in current hardware.

The setup

Hardware:

  • Processor: Intel Core i3 540 $119
  • Memory: 16GB DDR3 Corsair (2 x 4GB) $75 x 2 packs
  • Motherboard: Intel DH55TC $89 – Onboard video, Gigabit NIC, 4 memory slots, 12 USB ports, PCI slot.
  • Hard Drive: 500GB Seagate Barracuda 7200RPM $39
  • Network Card: Intel 10/100/1000 GT PCI NIC $29
  • MicroAtx Case & 350Watt Power Supply $39
  • Shared storage: Iomega ix2 Storage device $189
  • Network Switch: Cisco Small Business 10/100/1000 5-port $47

Alternate Hardware:

  • Processor: AMD Phenom Quad Core 9750 CPU $100
  • Memory: 8GB DDR2 (4 x 2048MB) $112
  • Motherboard: Asus M4A785-M $50

Software:

  • VMware ESXi 4.1 Update 1 Installable
  • VMware ESXi 4.1 Update 1 Installable on USB stick (Instructions: http://www.vladan.fr/how-to-install-esxi-40-on-usb-memory-key/)
  • VMware vSphere Client
  • VMware vCenter Server installed onto a VM
  • Microsoft Windows 2003 Enterprise Edition (less overhead / smaller install than server 2008)
  • Microsoft Windows 7 Professional 32-bit (choose any client OS you prefer)
  • Celerra NAS VSA

The process for registering and downloading VMware software remains unchanged. You can go to http://www.vmware.com and find numerous links to 60-day trials for all their software. ESXi installable is free to use which means your VMs will always run even if some licenses expire. I was interested in testing VMware View which included trial license keys and download links for vCenter, vShield Endpoint, View and clients.

There are two inexpensive routes for shared storage that provide NFS and iSCSI mounts. Celerra VSA (free from EMC to use) or a NAS device such as the Iomega iX2 or iX4. Other options exist but since I work with Celerra NAS I think it’s cool to have the software running in my lab. The Celerra VSA runs as a VM on one of your hosts. A better solution may be the Iomega iX2 which runs external to your ESX hosts. Shared storage is required for clustering ESX hosts.

How did VMware install?

Simple! Insert CD media, boot, install, repeat on all hosts. No issues with the Intel Motherboard. The H55/P55 chipsets work with ESX. USB, NIC, SATA, Video all worked fine. You have to go into the BIOS to enable VT and disable Execution Prevention Code. Clustering, HA, DRS, FT all seem to be working. You also have the option of running the hosts without a hard drive booting ESX from USB sticks. Each host would need to be configured with a NFS or iSCSI mount.

How to setup VMware

  • Install VMware ESXi onto each of the hosts.
  • Configure the root user password and host IP information
  • Install VMware Client onto a Windows machine. Point this to the IP of the ESX host.
  • Create a VM on the ESX host and install Windows 2003 or 2008 server.
  • Install VMware tools, OS updates, static IP, etc.
  • Install vCenter Server and let it install SQL Express (unless you want to first setup MS SQL Server)
  • Once vCenter Server is installed restart the client and connect to the IP of the Windows server.
  • Optional: Install Update Manager
  • Install Complete
  • Consider deploying the Celerra VSA virtual machine.

Example VMware Lab

The following image is an example of VMware ESX, VMs, vCenter & Celerra VSA setup on one PC. You can learn a lot about VMware with one PC but additional PCs are required for testing vMotion, HA, DRS, FT, SRM and other features.

Slide1

Two example setups using a second PC for vMotion or HA.

Slide2
Slide2

Two more PCs are required to properly test SRM.

Slide4

Contact me with any questions or comments!

Comments

  1. Chris says:

    Hi Thomas,

    I like your post and also the nice diagrams you used. What software do you use for your diagrams?

    I’m wondering with ESXi 4.1, is there a freely available client that can be used instead of vCenter? I’m still waiting on my hardware and I’m just wondering what exactly I will need to get in terms of software in order to make use of as much VMware techonoly without having to really buy licences as it is just for learning purpose that I will be putting this lab together.

    Thanks!

    Chris

    1. Thank you for the feedback.

      Head over to http://communities.vmware.com and look for PowerPoint templates. Most of the pretty graphics are used in high level design presos. As far as licensing, VMware gives out 60-day demo keys for all of their products (vCenter, View, vCD, etc). I suggest you check out the 60-Day VMware View demo (https://www.vmware.com/tryvmware/?p=view46&lp=1), it links you to downloads of everything you need: binaries, documentation and trial keys to get vCenter, View and vShield working. If your company is a VMware partner you can ping your rep about other ways to get license keys. Once your 60-day demo expires just sign up for another one with a new e-mail address.

      Do you need guidance getting ESX, vCenter, etc stood up?

      Good luck!

  2. avtobazar says:

    Nice topic – respect !

  3. Bob says:

    Hi – trying to install ESXi 4.1 with hardware. Intel DH55TC motherboard, intel i3-540 cpu, 16 GB crucial RAM, internal Western Digital 500 GB Sata (WDC5000AAKX).

    I am having problems with ESXi 4.1 install. getting error –
    vmkctl.HostCtrlException: Unable to load module /usr/lib/wmvare/vmkmod/vmfs3:Failure.
    I have enabled VT-d in Bios.
    Where in BIOS is the setting to disable DEP ?

    Thanks

  4. robert lee says:

    Hi – please ignore my previous post. I downloaded the latest ESXi 4.1 Update 1 installation file and is installing ok now. thanks.

  5. Dale says:

    Hi Thomas,

    Informative post, what kind of issues did you run into using the Asus M4A785-M motherboard with EXI 4.1?

    Thanks!

  6. Suresh says:

    Hi Tom,

    You have mentioned Asus M4A785-M in “Alternate Hardware”. This board uses SB710 Chipset which i could not find in VMware HCL. Only it lists SB700. Please let me know whether if i buy Asus M4A785-M and Phenom IIX6 1055T Processor, can i install ESXi 4.1?

  7. Kevin says:

    Hello,
    Nice blog. I am in a similar situation as you were when you started your lab. I have just been “move” to the Solutions Architect arena from a Wintel Security position.
    I wanted to get at least on VM host setup so I could better understand virtualization.

    I do have the USUS M4A785-m Motherboard with a Quad core AMD Phenon CPU, 4GB RAM. I have tries installing ESXi 4.1 and 4.0 on this systems. ESXi 4.1 gives me an failure error at the beginning of the installation and the 4.0 version gives me a “no approves NIC found.

    Just wondering what version of ESX did you install on your M4A785-M system ands were there any specials steps you needed to follow?

    Thanks

    Kevin

  8. Pradeep says:

    Do you have a step by step guide to installing ESXi, vCenter Server, VMware Client on a single machine?

    1. I do not, but that is a good idea, I’ll write one up on VMware 5 and post it here. Meanwhile, VMware does produce decent documentation on setting up ESX. Just go to their site and register for a 60-day trial. They have something similar to a quick start guide.

      The basic idea is as follows:
      1) Install ESXi onto the host (burn ESXi ISO to CD and deploy to internal HDD. You can also install to a USB stick using the instructions mentioned in the blog post above).
      2) Install Virtual Center Client onto another PC and point it at the IP address you assign to the ESXi host. (assume both Host and Virtual Center client are on the same network or cross over cable).
      3) From the Client you can deploy guest operating systems to the ESXi host, you’ll need 2, one for the data base (MS SQL), and one for vCenter Server. I recommend Windows 2008 Server.
      4) Once these two VMs are deployed and the OS is installed (enable RDP), you can use Remote Desktop to connect and install a SQL database to one VM and vCenter server to the other.

      VMware has a setup guide for this and I’ll try to write up a step by step shortly.

  9. dwb says:

    Hi Thomas,

    Thanks for the information. I have been trying to figure out a good solution for a cost effective home lab ESXi 4.1 server and really appreciated your post. I have used VMware Workstaion but have no experience with EXSi. I was hoping to to get your thoughts on using a quad core Intel Core i5-2500 Sandy Bridge processor instead of the dual core i3-540. First, will EXSi support all Core i3 – i7 processors? Would you expect to see any increase in performance in a home lab server with a quad core over a dual core? Would ESXi support Intel Turbo Boost?

    I welcome any input you may have.

    Thanks

    Dale

  10. Jeff says:

    Thanks for sharing, Thomas. I’ve been looking to do the same myself, so I took some of the hardware you listed and found some good current alternatives. If you have time to take a look, what do you think?

    1 I69-2300 – Intel Core i5-2300 BX80623I52300 Processor – Quad Core, 6MB L3 Cache, 1MB L2 Cache, 2.80 GHz (3.10 GHz Max Turbo), Socket …
    1 I69-2208 – Intel H55TC Motherboard – Intel H55, LGA 1156, Dual DDR3 Support, Micro ATX, SATA, HDMI, DVI, VGA, USB 2.0
    1 D15-1200 – DiabloTek PHD350 350-Watt Power Supply – ATX, SATA Ready, PCIe Ready, 20+4 Pin
    1 T925-1262 – Thermaltake VI5000BNS WingMA Computer Case – Micro ATX, 3x 5.25 Bays, 6x 3.5 Bays
    1 YYKM-740617166491 – Kingston HyperX – Memory – 16 GB : 4 x 4 GB – DIMM 240-pin – DDR3 – 1600 MHz / PC3-12800 – CL9 – 1.65 V – unbuffered – non..
    1 I69-6121 – Intel PRO/1000 PT Desktop Gigabit Adapter – 10/100/1000Mbps 2 T555-1338 – Transcend TS4GJF300 JetFlash USB Flash Drive – 4GB $13.98

    It looks like you can put together a hell of a lab box for $500 these days like you were saying, crazy. I’m going to just give it a shot with USB installable boot, and I have an Iomega ix4-200d I’ve been itching to use with an iSCSI setup for a while. Any feedback would be great, thanks.

    1. The new Intel Sandy Bridge Processors require a motherboard with the new LGA 1155 socket and Intel chipset. The motherboard you posted will not work with the new Sandy Bridge CPUs. When building a VMware lab I was attempting to save money, get 16GB of RAM and more CPU cores. Intel i5 provides 4 cores and AMD has a CPU with 6 cores. I have a couple co-workers running labs with AMD CPUs (See VMFORSP Home Lab Blog) It all comes down to the price per host you are willing to pay.

      The other main issue folks run into is the network card, slight differences in Intel NIC chipsets can cause issues. For a little more money I recommend the Intel EXPI9402PTBLK 10/ 100/ 1000Mbps PCI-Express Two Gigabit Copper Server NIC. This has 2 ports and it’s chipset is fully supported. The Intel PRO/1000 PT and GT do work and other VMware labs are running these cards.

      Iomega IX2 & IX4 are excellent shared storage devices, iSCSI and NFS work well. Again, having multiple NICs mean you can setup paths for storage access, management and VM traffic.

      Good Luck!

  11. Jeff says:

    Well crap, I would have yelled at myself years ago for making that kind of mistake, nice catch. :) I just don’t pay attention to the sockets like I used to. I’ll go give that dual port NIC a look too. Thanks for the response.

  12. John M says:

    Hi Thomas, I just came across your post here. Some really good stuff. I just thought I would add my 2 cents for people who may not have the necessary resources or funds to buy a couple of PC’s. I have succsefully set up an esx server using the VMware Desktop on my laptop. Granted, it is not super fast but for someone who is just learning virtual tech, it is a pretty good way to get started. Like everything else from VM you can get a trial licence. There are a couple of things you would need to know to install ESX on VMdesktop but after that it is pretty easy. If anyone would like some info on it you can leave a reply and I would gladly supply it. I am tasked with 2 projects in the first quarter of this coming year. Upgrading to Vsphere 5 and setting up and testing VDI. Any thoughts tips or pitfalls you could pass along?
    Thanks
    J

  13. Adrian says:

    Nice writeup! Sorry to dig up an old topic but were you able to get your DH55TC to boot esxi from a USB stick? I dont seem to have any issues booting other OS’s or booting esxi from the f10 but an automatic boot is no go…

    1. Yes, the USB stick is bootable. The host doesn’t have an internal HDD and BIOS is set to boot from USB first.

      1. Adrian says:

        Thanks for the the reply mate. Moved on to ESXi 5 and although you can directly boot to USB after selecting UEFI booting in the BIOS, unfortunately ESXi crashes on startup. Fortunately its very stable so not a massive problem, having to use the F11 boot menu for now.

        The good news is the onboard NIC works perfectly after updating the driver/PCI maps with Chillys vib.

  14. Prem says:

    Hi Thomas,

    I am new to VMware i have download ESXi5 and tryed to install on (x intel(R) Core(TM) i3-2100 CPU @ 3.10GHz, it give me an error message saying “No Network found” and says me to reboot again and again, can i know if this processor compartible for ESXi5

    Thanks,
    Prem