NSX Home Lab Hardware Setup

Hi Everyone, I know it has been a long time since I have posted (has it really been 6 months?!).

I promise will do my best to post on a more regular basis, especially since I have good hardware to play around with now.

A little update before we get started: I am loving my new role as a Systems Engineer for NSX at VMware and couldn’t be happier where I am at. Over the last 6 months I have had to transition from a deeply technical, daily grind position in tech support, to a more customer focused, business solution position. It has been a blast solving customer’s pain points around networking and security with our NSX platform; there is no other company who can do what we do with the NSX platform ☺ Anyway, let’s get to the point of this post.. my home lab buildout for NSX!

Over the last several months I was researching various hardware to use for my ESXi hosts. I looked at the Dell C6100, HP Proliant, Dell PowerEdge, and the Dell VRTX. After extensive research I finally settled on the following setup.
 

Hardware

 

Compute

2x Dell Powerdge R710

  • 2x Intel Xeon Quad Core E5540 2.53GHz
  • 64GB (8x8GB) 1333MHz 240-Pin ECC Memory DIMMs
  • Onboard Quad Gigabit 1000 Pro – 4 Ports Total
  • 2x1TB 7.2K SATA Hard Drive

compute
 

Storage

1x Drobo 5n NAS

  • 2x 3TB WD Red
  • Crucial MX200 250GB mSATA Internal Solid State Drive (For SSD caching)

drobo
 

Networking

1x Alcatel L3 switch OS6855-P14

  • 12 10/100/1000 ports 4 PoE and 2 SFP uplinks
  • AOS 6.4.6.125.R01

alcatel
 
The main reason that I went with the Dell R710s was due to the supportability of ESXi 6.0, the quad port nics, and it doesn’t hurt that they are quiet, in fact they aren’t much louder than my desktop. I opted out of the HP servers, for no particular reason, and the C6100 because I have heard that thing is so loud you can hear it a couple rooms down. The drobo NAS was a recommendation of my colleague. I opted to try it out over Synology or Qnap based on that recommendation and price as I wanted SSD caching and it seemed to add around $300 to the synology setup over Drobo. This thing is pretty slick as you can mix or match the drives and even upgrade an individual drive on the fly. As far as the Alcatel switch, one of my colleagues, Tom Rumland, provided this L3 switch for free a six pack of beer ☺ how could I pass that up?
 

Design Considerations

 
I decided not to nest my ESXi hosts on the R710s and instead use them as bare metal hypervisors. I went this route as I wanted to have good performance, wanted a more real world lab, and because of various VLAN considerations when nesting. As you can see, I only have one cluster that is a combination of Management, Compute, and Edge workloads, therefore both will be prepared for DFW and VXLAN.

Cluster-combo
 
Since I had quad port nics, I had enough flexibility to split out my traffic into 4 VLANs – Management, vMotion, storage and VXLAN. Each has their own vmnic, and the others are being used for standby redundancy as I ended up trunking all of the interfaces.
 
vlans
As far as the toplogy, I have started out very basic. I am running one Edge device and one DLR with OSPF. I am going to write more posts on configuring ECMP, multitenancy, and multisite/cross vCenter NSX, and even DR toplogies in the future but wanted to keep it simple for now.
 
toplogy1
 
The finished product is below! I know its not pretty right now, I still need to order a rack solution and haven’t decided what I want. So far everything seems to be running awesome! I have a ton of available RAM to use for future growth which I am not typically used to since I was always running on a desktop, or an internal cloud environment. If anyone has any questions about my setup, wants more details, or needs assistance setting up their own lab please feel free to comment below!
 
complete
 

Posted by:

Sean Whitney

2 Comments

  1. Heath Cejka -  August 7, 2016 - 9:51 pm 512

    Hey Sean cool post I’m really looking forward to your future post and congrats again with the new position! Just wondering where did you purchase most of the gear Amazon?

    Reply
    • Sean Whitney -  August 8, 2016 - 7:25 am 513

      Hi Heath, thank you very much! I ended up buying the servers off of ebay, the drobo off newegg, and the remaining stuff like msata SSD, power cords, ethernet, etc off Amazon. Amazon and newegg seem to have the same prices on most of the stuff.

      Reply

Leave A Comment

Your email address will not be published. Required fields are marked (required):

Back to Top