Source
r/homelab•
Mikevin

How would you set this up?

I have recently aquired the following hardware:

  • 2x HP DL160 G6
  • 1x HP Proliant DL120 G6

The DL120 has 3 disks of which two are in Raid 1, the other one functions as boot disk.

The DL160's both have 4x2TB disks.

I'm planning on running ESXI on all 3 nodes. The DL120 will have a management-role, might function as a firewall as well. The DL160's will run "production" VM's.

I'm not sure how to manage the storage though. The disk bays are maxed out so I'm stuck with this setup hardware-wise.

My question is: How would you configure this setup?

Would it be advisable to just run ESXI and let the raid-card do it's work or should I pass the disks through to FreeNAS, GlusterFS or something else? I want at least part of the space to be replicated between nodes as backup.

Disclaimer: I know this is a very general question, there probably isn't a "right answer". I'm mostly looking for some of your experiences and preferences.

Posted Feb 8, 2016, 3:17 PM UTC

1016
TechGy•

Personally, if it were me I'd get a VMUG advantage membership and do a 3 host cluster and run vSphere with Operations management and VSAN, but that's because I also run it at work and it gives me something to experiment with

3
MikevinOP•

I think I will use the free version for now and go with VMUG if I like it.

1
ianc1215•

Are you looking to run them in tandem? If so don't use VMware! The free licence prevents the use of HA.

If I was you I would setup each server DL160 to run something like proxmox or oVirt (The unbranded version of RHEV). Now the interesting decision. If you want each server to act on its own then I would use the DL120 as a UTM with Sophos, Untangle or something else like pfSense.

However there is another option. If you want both 160s to act as HA nodes then you should setup the DL120 as a SAN. With an SAN you have a few options depneding on how complicated you want to be.

Option 1 would be an iSCSI SAN, this is fairly easy to configure with LIO on Linux, FreeNAS, etc. You just need fairly high speeds links between each node, I would say multi-link gigabit at least.

Option 2 would be an FC SAN, basically the same thing but just a change in protocol and transmit medium. Since you have multiple nodes you would need an FC switch which will run you about 200 bucks. However the speeds are way better than gig ethernet and also its dedicated to just storage.

There are of course more options but I'll stop there.

Regarding filesystems you would have to use a "shared" file system like VMFS or GlusterFS. This is because when data is being written the other system would have to wait which won't work for a multi-node setup.

Changing gears a bit now. If you wanted to run a router for your network you could also put the router in the VM "cloud".

Hopefully this gives you some ideas on where you can go with this stuff.

3
MikevinOP•

I do want to run a hyper-visor on all of them, not totally sure if I really need HA features though. If I do I might go with the VMUG option.

I will read into proxmox a bit more, heard some interesting stories and it might add some nice experience as I've already played with VMware products.

If I'm going the SAN-route iSCSI would probably be enough for me. FC is interesting but a bit pricey(although it seems to be getting better).

So many layers, so many options. This will give some nice new challenges.

1
wolffstarrNetwork Nerd, eBay Addict•

Keep in mind that at our end of the supply chain (used gear), "getting cheaper" usually has a reason. Sometimes it's just age, but in FC's case it's because it's dying rapidly; 10, 25, 40, 50, and 100 gig ethernet speeds make FC and even Infiniband more of a hassle than they're worth. Good for us as toys, but the same thing that makes them cheap makes them not as useful in a skills sense.

1
Switchback77•

If this was my setup personally, I'd install ESXI on all of them and cluster the two 160s. Have the 160g6 run vsphere as a VM (and any other service related VMs) then just HA/DRS cluster the 160s. Of course this would require you to have a vSphere license and the non-free esx licenses.

2
MikevinOP•

Thanks! The licenses might be a bit too pricey for my usage. If I don't cluster them I wouldn't need a license right?

1
Switchback77•

Yea that's understandable. I'm sure something similar could be done with KVM or other free virtualization software.

I get VMware software (including vsphere and esxi) through my school for free. Another way you could go is VMUG which is $250? (Not 100%).

Without clustering (or vSphere for managing all the hosts at once) you should be able to get by on the free vmware ESXI license.

2
TechGy•

$200 https://www.vmug.com/p/cm/ld/fid=11

2
MikevinOP•

That's a lot better. I'll consider that, thanks.

1
nontoxicreddit•

For the DL160 you could run 2 arrays of raid 1. I don't have experience with software raids or anything.

You can also do raid 5 if you enjoy crying yourself to sleep

1
BenSBB•

Does the card support RAID10, if so id do that instead as you get a performance boost :)

1
nontoxicreddit•

Yes that is a good idea

1
MikevinOP•

It does. I might do that. This would limit my storage to 4TB if I were to replicate though. Maybe I should use raid 0 and raid1 seperately. This would give me 8TB of unsafe storage and 2TB safe storage over both machines.

1
MikevinOP•

Thanks! Any recommendation on what to use for replication?

1
nontoxicreddit•

I believe for any kind of replication through esxi you will need to have vsphere

1