Happy weekend y'all~~
Posted Sep 15, 2018, 7:01 PM UTC
#Physical:
Ryzen 1700, 16GB DDR4, 1x 500GB SSD
2x E5620's, 24GB DDR3, 4x 300GB SAS RAID6, 2x 1TB SATA RAID1
Pentium E2160, 4GB DDR2, 1x 500GB HDD
Hosts NGINX reverse proxy, main website, and OpenVPN.
#Virtualized:
Hosts multiple Minecraft servers for my friends.
Hosts a dedicated server for Space Engineers.
Hosts a dedicated server for Garry's Mod.
Hosts a dedicated server for ARK: Survival Evolved.
Hosts an instance of Rust that my friends play on.
Main Domain Controller with AD:DS, DHCP, and DNS installed.
Offline root certificate authority.
Host for the guacamole service.
Host for PRTG (free edition).
Hosts Bookstack website for homelab documentation.
Hosts and stores all my plex media.
Hosts an instance of phpIPAM for documentation purposes.
Provides a central network drive (like 10GB) that all accounts in my domain have access to. It's easier to use a network drive than a flash drive more times than not.
Will eventually be used to play Cards Against Humanity (Pretend You're XYZZY edition) with my friends.
#To-Do
Change hostnames to match the 'service Site# VM#' scheme. So for example I have MC01 now, and if I ever setup another one at a different location it would be MC21.
Setup a certificate authority on a new windows server VM.
Setup a samba share to hold basic program install files and scripts to run on fresh installs.
Purchase more RAM for HYP-V02.
Run ethernet so I can move HYP-V02 over into the storage room.
Purchase a decent UPS for HYP-V02.
Purchase a decent router and switch so I can work with VLAN's.
Setup Dynamic DNS internally and switch hosts over to DHCP so I don't have to keep setting static IP's on every machine.
Migrate game servers still on Windows 10 over to Windows Server 2016 Standard.
Configure a third DC on HYP-V02 and possibly decommission the old HP tower I have
Buying more RAM for HYP-V02 is definitely high on my priorities because I recently had an 8GB DIMM bite the dust on me. Luckily they are not very expensive on Ebay...
Peace out
How are you going to do the cards against humanity server?
PretendYoureXYZZY posted the source code on GitHub and I've found a few guides on how to install it, but I'm having trouble setting up the card pack database.
Oh nice
Did you get anywhere with the CAH server?
I attempted it a little while ago, but I can't start a game. It just says "Error: You must select at least one base card set" when you click on start game.
That's the issue I was having... I haven't really been messing with it recently
I guess they recently added a docker section at the bottom of their github page. I tried it out and it seems to work fine, but I am having trouble routing it through my reverse proxy now. It loads the static homepage fine, but as soon as click the "Take me to the game!" button it loads a white screen. The chrome developer tools are showing a SPDY error, while firefox loads the game.jsp page fine but you cannot set a username.
I seem to have it working now. So the CAH website is running using that docker container, and I have it routing through my reverse proxy using the following config...
I'm in the process of upgrading to 10 gigabit networking. So far I plan to move my two core servers (a DL380 G7 and a custom-built fileserver -- more on that below!) to 10Gig and stick to gigabit for the clients and WAN.
I've picked up a Dell PowerConnect 5524 as it was cheap and gives me 24 Gig-E ports and 2 SFP+ ports, which for now will be hooked up using DACs.
I plan to move to 10Gb fiber to all workstations by the end of the year as the budget allows, probably using a Ubiquiti 16 port 10GbE switch, with transceivers and pre-terminated fiber patch leads from fs.com as it's far cheaper than I thought (about 35 British Pounds for two transceivers and 100' of patch cable).
As a side note (and a 'has anyone else had this problem?') I have had to upgrade from my trusty Adaptec 5 series RAID cards (namely a 5805 and 51245) as I cannot get them to work in newer motherboards - they just cause boot loops on my Supermicro X9SCA-F board and the ASUS Z97-A on loan from my workstation runs them with an x2 link width. I have ordered an Adaptec 71605 which I am hoping will allow me to use the Supermicro X9 board in my file server. Edit, 9 days later: for anyone reading this - the Adaptec 7 series worked fine in the X9SCA-F board.
I just bought a second 5524 to get another couple of SFP+ ports. For the price they seem hard to beat. Now I just need to deal to the stock fans.
I had a somewhat similar issue where my LSI 9211-8i HBA card (IT mode) caused my Supermicro X8DTL-i mobo to not see or boot from USB after POST. To fix this, in the card's ROM utility I disabled it's boot feature (was set to "BIOS and OS", then "OS", but that worked only for a lil' while then I chose "Disabled" and now the mobo boots reliably). During this ordeal I learned this issue is a bug with Supermicro mobos, so it may related to your issue too (whether you're booting from USB or not). I read Supermicro released beta firmware for some boards affected by this ( you have to request it from them, not available publicly), but since the X8 and X9 series are discontinued I don't know if they received beta firmware that fixes this issue.
I haven't posted about my setup in awhile, but its very tame compared to most others on here.
Current Hardware
Upcoming Plans
I have plans to put my workstation in a rack-mount case (SM CSE-833) but I'm still awaiting additional parts for cooling and GPU mods.
The main upcoming project is my power and environment monitoring project "EMU" which is mostly built from industrial grade components and will have some open hardware (more than likely a raspberry pi). The system will monitor 3 "lines" (incoming cabinet feed, UPS & non-UPS) of power using a 3 phase meter (Satec PM172EM). The meter will connected to a raspberry pi over serial RS485 (Modbus Protocol). There it will translate the modbus data into snmp data that will be read by LibreNMS and it will also store it in its own database and have its own web interface to only view current parameters and values. This system will also power the fan on the cabinet as well.
No other changes are expected.
Huge downsizing for me within the last 30 days...SWMBO decided the heat generated by the machines (living in South Florida) was not appropriate any more, as well as the noise adjacent to the toddler's room :(
Currently running
Powered Off but on it's way out
Retired within last 30 days......
Sorry for your loss...
R.I.P.
I haven't done much with my lab recently. The biggest change recently has been moving my 24 SSD array from a Norco 4224 to a SuperMicro SC216. Looks much sexier now.
My scheming from when last we spoke appears to be paying off.
I've taken a single Supermicro X9SCL-F board and put it into a server that I'm currently using as a super-simplified SAN - CentOS on a small SSD with a ZFS mirrored vdev pool totaling 2TB for VM storage. I've tested the Dell 0KJYD8 cards that I had lying around with some SFP+ receivers that I bought on eBay in various configurations, and everything seems to work well. It looks like it's time for me to move on to Phase 2 of my plan :)
In preparation for Hurricane Florence (I live close to the east coast) I also went ahead and splurged on new batteries for all 4 of my UPSes - two Cyberpower 1500PFCLCD's and two APC Back-UPS Pro 1500's. I think, once I get the proper cable from Amazon to tell the APC's that they have new batteries and thus report an accurate remaining time to me, I will use those in my homelab, particularly because I can purchase battery expansions for these models to get even more runtime out of them. I'll likely use the Cyberpower UPSes for mine and my partner's desktop rigs. This was a relatively expensive purchase (compared to how much I've spent on the rest of my homelab), but it's definitely going to be worth it to be able to actually trust my UPSes in case of brownouts/blackouts going forward.
With all of that said, here's everything that's currently in my homelab:
Working on moving all Windows servers over to Linux.
Currently stuck on getting persistent storage and network storage to work with Docker.
Running DoD 5220.22-m drive wiping on all my old hard drives (about 20)
Seems like a good use of the weekend.
Nothing like making sure your data is kaput before you let it out of your sight.
I finally took the plunge! I had been deliberating on getting a server at all, and once I was already buying a new UPS, I figured it was probably time to get knee deep in server stuff (seeing as how my month has freed up tremendously with a bunch of free time after work)
I'm mostly looking to start getting into FreeNAS but I'm also trying to build a solid foundation of understanding how servers work, so I figured spending an extra $20 or so to get a better machine than I could get from just some synology NAS was going to be a better deal. Plus, the CPU I found is about on par with my 4690k in my main machine. I could probably host a minecraft server or something.
PCPartPicker part list / Price breakdown by merchant
| Type | Item | Price |
|---|---|---|
| CPU | Intel - Xeon E3-1220 V3 3.1GHz Quad-Core Processor | $85.00 @ Ebay |
| CPU Cooler | Noctua - NH-L9i 33.8 CFM CPU Cooler | $39.95 @ Amazon |
| Motherboard | ASRock - E3C224D2I Mini ITX LGA1150 Motherboard | $219.60 @ Amazon |
| Memory | ADATA - XPG V1.0 8GB (2 x 4GB) DDR3-1600 Memory | Previously owned |
| Storage | 2X WD Red 4 TB HDDs (5400 RPM) (Raid 1) | $240.00 |
| Case | Silverstone - DS380B Mini ITX Tower Case | $149.99 |
| Power Supply | Corsair - SF 450W 80+ Gold Certified Fully-Modular SFX Power Supply | $85.43 @ Amazon |
| UPS | APC - BR1000G UPS | $132.29 @ Amazon |
| Total | $952.26 | |
| Generated by PCPartPicker 2018-09-17 11:57 EDT-0400 |
That's pretty much it. Parts have been bought, shipping is expected to occur next week.
I'm hoping to store my music production stuff on my server, and I may or may not end up taking my current spare 500 GB SSD and using that as a separate drive on the server, hooking all my music files up to that instead of google drive, and generally having more control over my data.
I dunno, thoughts, comments, concerns?
I'll probably be seeing more of you guys over the next few weeks as I learn about owning and operating a server for the first time.
That's an excellent start. If you haven't already, pick up a couple USB memory sticks. FreeNAS will automatically install to both in a RAID1 configuration and you can keep the OS separate from your data volume. And that case is sick...lots of room for drive expansion but still in a small package.
That's an excellent start
Thanks! I have some experience buying and building PCs for others so I hope I didn't fuck something up that majorly.
FreeNAS will automatically install to both in a RAID1 configuration and you can keep the OS separate from your data volume
What's the point of this? Does it really matter if I run the OS off of an SSD vs USB drives? Are there specific advantages that I should be aware of?
Just added a MacOS Caching Server to my home(lab) environment. I didn't realize just how helpful one would be. With as many iOS and MacOS devices I have running at home, plus the work devices I also carry, not needing to repeated pull data and iCloud content from Apple's servers is a nice savings for my monthly data cap.
MacOS Caching Server
Ohhh, this is a good idea. I need to add this to my never ending list of stuff to do. Probably don't has as much Apple stuff as you, but it does seem to be permeating more and more of my existence.
So just setup a Mac OS VM and everything else in terms of sharing just works?
How did you get caching working, and what version of macOS? I have macOS running in a VM but caching doesn't work as it detects it as being a VM. Tried steps to make it seem like a real one but to no avail.
I'm doing my first complete (or very nearly complete) network stack rebuild. I picked up a second DanTrak Skeletek 28U 4-post rack to replace the enclosed 16U rack I've been using for my "production" hardware. The ancient PIX 525e failover bundle is getting replaced with a slightly less ancient (but no less ridiculous) ASA 5580-40. The Cisco 3745 router is being replaced with a 3945, and the NM-NAM network analysis module is being replaced with an SM-SRE-910-K9 running Cisco Prime NAM.
I went a little crazy with the UPS after years of fighting with smaller consumer-grade UPS models. I picked up an APC SUA3000RMXL3U with an external battery pack (and my back is still killing me a week later) and ran a 30-amp circuit for it. I also replaced my absurdly old AP9211 PDUs with AP7901 PDUs. The AP9211s were still rock solid, but I wanted something with 20 amp capability and a slightly more secure network interface. I picked up the AP9631 SmartSlot card for the UPS and a couple of thermal/humidity probes. I'm planning on setting up a fancy Grafana dashboard, so I'll have to figure out how to integrate these.
ASA 5580-40
I always wanted a 4U firewall. Sure that thing is a wonder on the power consumption. Still, probably looks kinda cool.
My lab rack is up to ~1,640 watts. Can't wait to see the hit to the power bill.
The ASA is absurdly loud and puts out quite a bit of heat. Fortunately, this rack is tucked into a part of the basement that is fairly well insulated, or my wife would have my head.
Pfft, at that point, might as well just use the thing to help heat the house. No point running other heat sources if you've already got one going.
Totally not speaking from experience or anything....
Physical:
Virtual (VMs and LXC containers):
Sounds like a perfect excuse to get an HA cluster going!
dafuq! great man!
there are people in this sub that could set up a 42 rack for the same stuff
It is okay, on big iron anything below 80% was worrying because of underutilisation of CPUs.
I bought a used R710 earlier in the month for $100. I'm planning to add a few things before I put it into production.
Should be a fun project to get running, and I can migrate FreeNAS off my existing desktop computer onto some better hardware with more processing power and ECC RAM.
###Current Hardware
###Future plans
ESXi 6.7 Lab:
Other:
Plans for upgrade:
So far this homelab has taught me more than a book and landed me a job at a ISP/MSP. Looking forward I wanna do Lab 2.0 with r720s but the price to upgrade is too high right now. Really enjoying what I see here and it makes me so happy to find a community of people who share the same love and dedication to hardware/software as I do. Can't wait to see what everyone else post here!
and hardware_jones really liked your formatting, so I "borrowed" it :)
Just getting back into homelabbing as a freshman in college, which is definitely not the best decision for my wallet, but that's alright. Also, I'm on mobile, so sorry for the formatting.
Current:
Dell R710 SFF
2x Xeon X5570
48gb RAM
2x 60GB SSD for Windows
1.5TB~ usable space
Windows Server 2016 Datacenter Running:
AD
Plex
VPN
Minecraft server
HP Procurve 2824 Switch
Some Trendnet Gigabit POE switch, which I plan on replacing with a nice Aruba switch I got under warranty
Future:
Currently running esx 6.5 u2:
Cisco C240 M3 (2x E5 2697 v2, 320gb RAM, Cisco 2x 40gbit VIC, 2x 1200w psu)
-VSAN: 60gb corsair FORCE ssd 2x hp 300gb sas
Cisco C240 M3 (2x E5 2697 v2, 320gb RAM, Cisco 2x 40gbit VIC, 2x 1200w psu)
-VSAN: 60gb corsair FORCE ssd 2x hp 300gb sas
Cisco C240 M3 (2x E5 2660 v2, 196gb RAM, Cisco 2x 40gbit VIC, NVIDIA GRID K340, 2x 1200w psu)
-VSAN: 60gb corsair FORCE ssd 2x hp 300gb sas
Shared Storage for vmware (storage spaces direct) Windows Server 2016 Datacenter:
Custom build (E3 1245 V2, 32gb RAM, 4x 40gbit melanox card, 6x 600gb intel dc ssd, 3x samsung 120gb nvme)
General Storage (storage spaces direct) Windows Server 2016 Datacenter:
Custom build (E3 1245 V2, 32gb RAM, 2x 10gbit hp nic, 3x 120gb inland pro ssd (cache) 9x 2tb wd red)
PFsense Router:
Dell Precison T1700 (E3 12v1 V3, 32gb RAM, 2x 200gb micron enterprise ssd, 2x 10gbit hp nic, 2x intel 1gbit nic)
Nutanix CE Cluster:
HP 800 G1 mini (i5 4590t, 16gb ram, 128gb samsung nvme ssd, 500gb seagate hdd 1x intel nic)
HP 800 G1 mini (i5 4590t, 16gb ram, 128gb samsung nvme ssd, 500gb seagate hdd 1x intel nic)
HP 800 G1 mini (i5 4590t, 16gb ram, 128gb samsung nvme ssd, 500gb seagate hdd 1x intel nic)
I'm currently trying to migrate to a VyOS HA solution from a hardware router/firewall to reduce the amount of power used and, by moving to a lower powered switch, be able to run another server. Are there resources for VyOS' commandline other than their wiki that come with examples, especially in regard of PBR and HA setup?
[deleted]
Is there an expected date for the release of v1.2? There's a number of things I'd need from [https://wiki.vyos.net/wiki/1.2.0/release_notes] here.
[deleted]
[deleted]
Currently running under ESXi 6.5:
Configured, on standby:
Misc. running:
Misc. not running:
Buying parts for:
A few weeks ago I bought, configured and installed Brocade ICX6610 (1/10/40GbE) and ICX6450 (1/10GbE) switches, along with 4x Mellanox 10GbE and 2x Mellanox 40GbE HCAs, all connected via DAC. I'm slowly transferring all networking from a fully populated Nortel 5520 over to the Brocades, and moving all 13 cams to their own server and subnet. At the same time all servers have been moved to 10 or 40GbE and ESXi re-configured so that moving VMs around is much faster than before... Still need SSDs to replace all spinners but the cost is too high at the moment.
Next up is moving everything to ESXi 6.7 and talking myself into a second VMUG license after I try Proxmox.
https://i.imgur.com/jD0hNKJ.jpg
/e: typos
That is insane bro.
This guy homelabs.
Can you expand on the r pi as a lightning detector? What is that setup?
Here ya go:
https://shop.switchdoc.com/collections/weather
I've a Davis weather station online for years; the lightning detector will make for a nice upgrade.
Any automation? Like automatically shutting down non-critical systems when a strike is less than 5 miles away?
The thought has crossed my mind. This step is to find out how accurate it is, and to get the notifications in place. I'd be happy if it can tell me when a cluster of high density strikes is closing fast; Plan A is to (i) run out to the power pole and trip the breaker, separating me from the 2 miles of aerial copper conductor run perpendicular to the normal storm track, (ii) run to the shop and trip the incoming 70A breaker, (iii) run back to the house and trip the incoming 200A breaker, (iv) initiate orderly shutdown of UPS clients. This can be done in 5-6 minutes, with 5 dogs barking in the background as dead UPS beeps argue for attention with a whole bunch of holyfuckthunder.