Acceptable top level responses to this post:
What are you currently running? (software and/or hardware.)
What are you planning to deploy in the near future? (software and/or hardware.)
Any new hardware you want to show.
Previous WIYH:
Posted Mar 15, 2019, 12:10 PM UTC
u/Cosmic_Failure is the new u/MonsterMufffin I take it? Props to you/her/him and whoever keeps this /r/ alive.
The lab used to be more extensive, but unemployment and cost savings have whittled it down some.
Hardware:
2x SC2600CP2J motherboards with 2x E5-2670 v1 & 128GB DDR3 RAM each
8GB RAM Celery(tm) laptop running vSAN witness host
2.5TB SSD Storage between the two hosts running in RAID1 vSAN
LB6M 10GB switch
HP 1920-16g 1GB switch (not sure why Amazon sells them for 500 USD, considering I bought it new for ~160 USD a decade ago)
Software:
Nested ESXi lab - need MOAR RAM, always and forever!
Storage server running a no longer maintained FlexRAID setup with 2x parity - 58TB spinning disk storage - time for a refresh, it just sucks transplanting existing NTFS HDDs to a new filesystem. Blah....
Cloudron
Docker host
Guacamole
OpenVPN
RetroArch Web emulation
Plex (Obligatory)
Sandstorm
Tor relay
Univention CS
YunoHost
Archiveteam Warrior appliance
BlueIris DVR
Windows VM used as a jumpbox and Visual Studio IDE environment
pfSense VM (provides Internet access to household)
Torrent host
Ubuntu 18.04 desktop
2x Veeam Community Edition VMs
VCSA appliance
several Windows and Linux templates ready to go for cloning purposes
Pretty simple compared to the more intricate setups I love reading about on this sub.
The entire mod team keeps this sub alive. These megathreads are just posted by any one of us, whosever gets to it first really.
Glad to hear it. It's been a bit quiet on the SoftwareDefinedMuffins front, so I had assumed you passed the baton.
I still need to get my VPN setup going on pfSense. I've your blog post bookmarked. Cheers, glad to see you're still around.
No problem dude, just passing on credit where credit is due.
I've been really busy at work and in my life recently and haven't been able to sink as much time into this subreddit and my blog as I'd like, but I have plans for new posts and hoping to pencil in some time soon!
Have a good one brah.
You bum. What work and life things could possibly be more important than this subreddit?!?!!!? :)
Love the fact you got the 1920 switch, I used to swear by these, the most underrated switches I've ever come across.
It's a wonderful unit that's served me well for many years. L2+, easy setup with webgui, functional enough CLI, and mostly quiet. I put a piece of tape on the fan exhaust and it made it noticeably quieter. (I know that's crazy but I use and abuse my hardware, and at home noise matters more than worrying about potential failure of Enterprise level hardware.. You should see what I've Frankensteined into my 4U and LB6M... 0 failures yet, apart from cat jumping off the switch and throwing it on the ground, where I found it still chugging along)
Starting this month im getting back into the home lab money pit. after selling/giving most away of my lab in 2013 due to energy consumption and scaling down to a single Synology DS1813+, i got a change to get some ( IMO ) really nice hardware and get back in to it
got myself a nice little setup consisting of:
but being the a bit of a neat nerd i REALLY want it not only to work well but also look nice and be somewhat quite enough so i can still work / play in the same room.
so i still need to find myself:
after thats been taken care of its build/config time.
planing on becoming VMUG advance member so i can use all the vmware products in the home lab
still debating to myself how i want to realize the following
after thats all done im planning on using it for:
the setup will most likely (ofcourse it will) also replace the electric space heater i have in my Cave so i will use that argument to convince myself this is a smart thing to do.
so yeah thats the stuff im starting on in march
also this i my first ever post on reddit YAY!
Would look at radar and so are in place of sick ears. Your setup sounds great though.
DS1813+
keep the Syno for 'production' stuff (important files, photos, etc.) you dont want to put that on you lab.
When you do pihole (which you totally should!) look at doing the openvpn install alongside it. I just did it on mine and I love it.
IBM 36U cabinet (used to hold a bunch of ancient AS400 gear). Converted the power rails on it to run on 110, runs on two dedicated circuits out in my garage.
HP ProCurve 24 port managed switch (1GB... don't remember the model off the top of my head, but it does VLANs and LACP which was all I needed. Originally, I was using a 1U pizza box server running pfSense but it died)
3 x HP DL380 G6. Dual x 5670 + 64 GB RAM (yeah, I need more memory)
1 x IBM x3650 M1. Don't remember the CPUs, but its 2x4 and has 48GB of RAM
Old IBM KVM + 8 port KVM switch. Gets the job done...
IBM DS4000 14 disk Fibre Channel Array with array extender (also 14 disks) but it's loaded with 36GB 15k disks, so its only about 1TB. It is fast though.
the HP servers run ESXi 6.5, the IBM runs FreeNAS and acts primarily as an iSCSI target. Has 12TB of storage.
in my vCenter, I run a few dozen VMs, primarily CentOS 7 for hosting various applications/services. I have some actual RHEL, but I'm limited on licenses so I stick with CentOS for the most part. Ipsec VPN, PLEX, PlexConnect proxy, Active Directory, SCCM, Grafana for dashboards, Spacewalk, AWX, TikiWiki, a Minecraft server for my kids, a Linux From Scratch build host, Netatalk (open source AFP implementation for Time Machine backups). A couple jump boxes for managing various environments. A macOS VM that I use specifically for iMessage. you name it... if its a service that can run on a VM and it's marginally useful, I'll probably throw it in there at some point.
Future plans are to deploy vSAN, primarily for learning it, and eventually upgrading to 6.7. I may one day upgrade some of the Windows VMs from 2012R2 to 2016, but... AD works as advertised and I'm a Linux guy first. RHEL/CentOS 7 does everything I need other than Active Directory. I'd like to get an nVidia quadro card so I could play with PCI pass-thru at some point.
Can you provide any information on how you got iMessage working in a VM? I've never had any success with it and would love to get it working in my lab
yes, it's actually not that hard. You just need to add and remove a few entries to the .VMX file. You're accomplishing the same thing as you would with injecting hardware IDs with Clover, so I'd recommend using Clover Configurator to generate the values. You'll need a fake serial number that doesn't correspond to a real Apple product, you'll have to choose a model of Mac to emulate (FWIW, I always use 'Xserve' for VMs running in my lab so as to visually differentiate between them and physical Macs in my home network). You'll need a valid hardware MAC address for whatever device becomes en0. Here are some of the values you may need to edit
ethernet0.virtualDev = "e1000e"
ethernet0.addressType = "static"
ethernet0.address = "00:25:bc:00:XX:XX"
ethernet0.present = "TRUE"
board-id = "Mac-F223BEC8"
hw.model.reflectHost = "FALSE"
hw.model = "Xserve3,1"
serialNumber.reflectHost = "FALSE"
serialNumber = "CK9070XXXXX"
smbios.reflectHost = "FALSE"
efi.nvram.var.ROM.refletcHost = "FALSE"
efi.nvram.var.MLB.reflectHost = "FALSE"
efi.nvram.var.ROM = "284B53XXXXXX"
efi.nvram.var.MLB = "CK9060XXXXXXX"
I set this up a while ago so I'm a little fuzzy on the details, but I think these are the only edits you need to make (other than removing the default entries that these settings override). I can't find the original guide I followed but there are a few videos on youtube explaining how to do it on VMWare Workstation, but I can assure you that the same essentially works on ESXi.
I hope you can figure it out. Good luck!
This is super helpful, thank you!
This week was spring cleaning in IT, and as part of that my boss let me take a unused mobile rack, an old C2960-S that was barely used, and an HP DL360 G6 that was previously used as an '03R2 Print Services server.
DL360 G6 Current Specs:
Planned:
I don't currently have plans to install a PCIe card or two at this time, as I'm not sure exactly what I want to do with it yet. I'm thinking I'd use it to learn NixOS and Docker, while running a few nice-to-have services (caching for Steam/UPlay/Origin, network shares, etc.).
One question for other HP G6 owners: I'm currently running v3.x of the P410i's firmware (which maxes out at 2TB/disk), and I would like to upgrade that; do I need to install each firmware update sequentially, or can I just jump to the latest revision? After that, are there any caveats I should worry about?
"One question for other HP G6 owners: I'm currently running v3.x of the P410i's firmware (which maxes out at 2TB/disk), and I would like to upgrade that; do I need to install each firmware update sequentially, or can I just jump to the latest revision? After that, are there any caveats I should worry about?"
You can upgrade directly to 6.64 without problem. You can also snag a SAS Expander card and second SFF disk enclosure on eBay for next to nothing, relatively. HP 628974-081 (or compatible) 16GB RDIMMs are inexpensive too; you may also be to pick up HP PC3-8500R 16GB RDIMMS or even 32GB RDIMMS (!).
"My heart says X5670"
Lurk on eBay, waiting patiently...I saw X5675 going for $18/each the other day, though that is not the norm.
"4x 2TB Seagate Barracuda Pro 2.5" 7200RPM SATA Drives (I don't necessarily need flash, especially as the P410i operates at SATA II speeds)"
If I remember correctly, SATA runs at 3G on the P410 and P410i. SAS will run at 6G though; you can mix and match drives with this controller with no issue. I would use Western Digital Red 2.5/SFF NAS drives, though. I have used both 3.5/LFF and 2.5/SFF Western Digital Red in both Gen6 and Gen8 ProLiants quite successfully - and that's without the fan issues that some people have reported. Of course, YMMV.
I personally upgraded my P410 (non integrated) from a version from 2010, to the latest version (I believe it was 2016), with no issues. AFAIK, it is the same firmware as well for both. You can use the SPP editions if you want it easy.
It is indeed the same version. The P410i is just the P410, integrated on the system board. Firmware, cache modules, battery packs, SAS / SATA cables, the Expander...all of them work on both just fine.
This also holds for the P420 and P420i.
Perfect, just making sure :) Albeit, the P410i has worse performance (if I remember correctly).
But yeah, I updated mine from rock old to newest. Shouldn't be an issue.
By the way, I find it easier to just grab the files for the ROM BIOS, P4x0 controller, and iLO instead of dealing with SPP - iLO firmware can be upgraded over the network, the ROM BIOS using a bootable (FreeDOS) USB flash drive, and the P4x0 controller using a bootable (I like Debian Linux :) USB flash drive.
you should be alright to just run the latest SPP iso. I've never had any issues between firmware versions.
My network is currently running on an old netgear switch, I got a brand new ARUBA from my work and I want to use this new one for "VLAN"ised my network, so I could do a greate PfSense cluster.
Also, I want to run a syslog server (maybe graylog) and try SolarWinds service
Currently running 2 nodes with proxmox, working on building out a monster 3rd quorum member/iScsi host for god awful amounts of storage.
Reusing "prosumer" hardware stuffed with RAM.
Model: OptiPlex 7010
Memory: 23.4 GB
Processor: 4 Cores Intel Core i3-3220 CPU @ 3.30GHz
11 TB online
Model: HP ProDesk 600 G1 SFF
Memory: 31.3 GB
Processor: 2 Cores Intel Pentium CPU G3420 @ 3.20GH
1.2 TB online
I see that virtualization is quite popular here. Where I work containers are the stuff. Is here someone who is running container orchestration like kubernetes or openshift on hardware in their homelabs?
I feel like container orchestration just isn't as popular for homelabs other than learning. Honestly the built in container managment for OMV or portainer is enough for most people.
I run containers in my homelab so I can keep all of my media server stuff on one VM and I don't have to go through the installation process because I'm lazy but other than that I don't see the need for it. If someone else thinks differently I'd definitely be interested to hear why.
I run containers k8s to orchestrate containers for Plex, personal VM management portal based on OpenStack, etc.
rebuilding my filer server i put together with spare parts. c drive died in such a away it shut of pc. also i think also i need to replace psu soon to.
How are you running your file server?
been trying omv,ubantu,win 7. this is going to be a file dump server.
Just picked up my new KVM from the post office :D It is a Raritan Dominion KX3-808, replacing an old Trip-lite 16-port KVM console (15" screen).
Also had to add another switch, decided on a 24-port non-poe unifi switch, it is joining a 24port POE (250w) and 16xg :)
Might have to replace the batteries and install my second UPS soon, power bill keeps growing on me. To qoute the spiderman movie, "with great power comes great electricity bill"
Did you find a deal on that kvm or did you end up paying 5k? How is the kvm working out for you?
I got a deal, paid about $500, including 4 D2CIM-DVUSBs.
It it great with the local monitor. It is a bit annoying that the mouse does not always sync when using it trough Chrome, but that might be down to some settings.
I bought a couple of Hikvision PoE cameras off eBay a while ago but setting them up is queued behind a whole lot of hairy yaks. I'm hoping to make progress this month!
I'm a bit reluctant to plug in reused hardware into my network as it stands. To feel better about it, I want to set up an isolated network to plug things into that will keep them deaf, dumb, and network-blind by default. That should be mostly a matter of Unifi-USG configuration and a separate vlan, but I'd also like to have monitoring of what's happening on that network, both at the network level and DNS level, just in case the cameras arrived pre-malwared (insert image of alien parasite scrabbling madly at inside of glass jar). This might also need per-device vlans, if I can't work out how to isolate potentially mutually adversarial devices on the same vlan, at which point we're talking 802.1X and Radius-assigned vlans.
From what I understand, USG firewall rules can log, but I need to set up a syslog receiver to handle them in some useful way. Ideally it'd feed into a long-term database to track long-period behaviour, but also advertise a real-time view (maybe via MQTT?) so I can tweak firewall rules reasonably interactively to make a new device work (not relevant for this camera, but that should be the process for setting up new IoT things in general). I'll also need to find a logging DNS blackhole that can do permit/deny/log rules per device.
Then if I have permit rules on the firewall for quarantined devices, I probably also want netflow tracking of allowed traffic from the quarantined vlan, which will require another collector/database/etc, but this is getting above and beyond what I need for the cameras. Replicating all the above for wifi-based IoTs is going to be interesting too.
I'm starting out with a Unifi network (USG, switches, and APs) and a NUC running Docker containers. This will be fun! :)
Check out graylog, I've used it with ubiquiti devices before with some success. There are filters you can set to make it easier to read too.
Thanks for the Graylog tip! I thought about it for a few minutes, but decided I'd rather build things out of pieces. Reasons I came up with to justify the decision that I really wanted to make anyway:
By way of yak-shaving update, I have the unifi to syslog to MQTT to BigQuery path dockerized and running happily, and I'm now poking at CoreDNS to make the DNS logging blackhole. Logging is going to be via dnstap to MQTT to (obv) BQ again, so that's nice, but I've given in to the urge to add pi-hole style blocklists into CoreDNS (not directly supported, but there's some out-of-tree plugins) for my non-quarantine networks, and I'm valiantly struggling with the urge to also add an IDS to my Docker network, since I'm attaching so many chatty containers to it.
I noticed in the process of setting all of that up that there's some troublesome wireless devices that keep falling off the network and rejoining (yay logs!), and also that my thermostat and SDR feeds have fallen over and need dockerizing and replumbing through MQTT as well, so I should make time this weekend for that too, and maybe add some alerting to Prometheus to catch future recurrences.
You'll note that I still don't have any cameras plugged in, nor a quarantine network to plug them into >_<
Currently running some Linux machines for the moment. I just nuked some SSDs and an HDD by accident because typed /dev/sdc insead of /dev/sda in the command line. But I had a backup, what matters ? On hardware level, I run Debian stretch on not server parts :p :
Also got a NAS for my daily backups and some other random stuff I might end up storing on the NAS.
Got some stuff running on my main server:
I haven't finished restoring services to up and running, but I'll get things back when I finished messing up with BTRFS commands that I'm slowly but surely discovering ^^
I don't have many plans for now in terms of deployment. I'll first finish restoring all the tings before I can think about some other stuff and move on... ^^
First "Lab" currently running i7-3770, 2x500gb SSD's, 2x 2tb HDD's, 16gb DDR3 and a PCI-E Intel gigabit adapter.
No idea what to install on it, was originally thinking of using OMV on it though i have terrible upload so it would be useless for remote access. Thinking of putting running Plex/Sonarr on there but not sure what else i should put and what main OS i should use?
Just setup a 3 node nutanix CE cluster :)
My lab:
Running on Proxmox 5
R210II -> Pihole (Debian container) / Pfsense
Dl360 G7 -> Empty ( looking for ideas around here)
Rpi3 -> OMV
Netgear router -> DDwrt with 2 Airport Extremes
Future:
Patch Panel
Gigabit Switch
Replace DDwrt with Pfsense
Replace OMV with FreeNAs
Get R720 ( dunno what for but i want it :-)
I've shipped my last server away, sold all of my local gear. I'm moving to a place with solar and wind power exclusively, no grid tie ins, so the lab has to move.
I'm now left with three servers, one cloud box.
Personal Hypervisor: 64GB DDR4 I7 7700k 4x10TB drives 2X512GB NVMe Unmetered bandwidth on 2G port Ubuntu 18
Monitoring Station: 16GB DDR3 E3 1230v2 80 GB SSD Unmetered bandwidth on 1G port Ubuntu 18
Eco Hypervisor (experimental) : I7 7700k 64GB DDR4 2x10TB 2X512GB NVMe Unmetered on 10G port Openshift
Cloud: 8G memory, 25GB SSD, 1 IPv4, unlimited 500mbit. Used for grafana, backup monitoring network.
Playing with a few servers each with SSDs in the lab, what's the best way to do clustered storage? Right now looking at VMware VSAN, Starwind Virtual SAN, and perhaps Proxmox w/ Ceph. Anything else I should be researching? Hoping to do compute+storage on each node.
I just rebuilt my homelab this weekend and it's been great! Everything is so fresh and easy to document. It's not complex (just one box and 2 networking devices) but I love it and it's easy to expand.
Hardware:
Whitebox Server:
Networking Equipment:
Edgerouter X for all of my network traffic with a guest VLAN
Unifi UAP-Lite for wifi
Software:
Proxmox as my hypervisor using ZFS to have the WD Reds in a raid0 config
(none of my data is essential other than what I back up to the external drive)
Proxmox VMs:
OpenMediaVault: managing my network shares, docker containers, and backups
Unifi: managing my access point
Caddy: reverse proxy to access my services remotely
Pi-hole: ad blocking
OpenMediaVault Docker Containers:
Transmission-OpenVPN: Bittorrent client that uses my PIA VPN for downloading files
Plex: Watching TV and Movies on my Chromecasts/Android TVs
Sonarr: Automatically downloading TV Shows I enjoy when they come out
Radarr: Automatically downloading movies I enjoy when they come out
Nextcloud: Share files with my Mom across the country
Mariadb: Database for Nextcloud
Organizr: Nice interface for accessing all of my services
Guacamole: Remotely access my desktop when I'm away from home or too lazy to walk across the house
Nessus Scanner: Scan my machines and VMs for vulnerabilities (still learning how to use this one)
It's not a very hard setup but I enjoy it a lot and it makes life easy. What else could I ask for?
Just started my homelab this last Saturday. I work in IT, but don't really know networking yet (just promoted from a level 1-->level 2 tech), so this has been a learning experience.
Deploy next:
I installed pfSense baremetal. Should I have installed it as a VM so I can also run a print server on that same box? Not sure if that would've been good to do. I don't feel like I need or want a full Windows server just for a single printer for up to 5 people.
I do have another box with Kodi installed that I haven't used yet. Maybe I should virtualize everything on that?
What is your favorite software/service you're running that isn't Plex, radarr, or sonarr? I'm looking for ideas :)
Been getting on the document management system train with teedy. Also been setting up and planning my automation of nessus community edition. We use it at work and I feel like I should know more than entering his hostname and clicking scan...
isn't teedy cloud-hosted? or is there an on-prem solution?
There's an on prem docker image.
Woah! It's your 4th Cakeday chesser45! ^(hug)
Any resources or guides for Nessus to share? I was looking at setting that up as well.
Sorry for the late reply..
we use security center at work and its locked down to mostly just host scanning. At home I setup the free home version to fiddle with. You can only scan 16 ips per scan but I guess just scan a couple sets of your IP range at a time, and some features are locked out. Good for a free way to keep core infrastructure patched or see if your images are being patched when applying KBs in the lab.
Link:https://www.tenable.com/products/nessus-home
Looks like they have some free labs I haven't touched here: https://www.tenable.com/education/on-demand-courses
Cool, thanks for the tips. I set it up last night and boy, it seems thorough even with just the basic test. Any advice on what I should have it look for?
I'm just running the basic network test for now on only the local 192.168.x.x IPs used by the server, LXCs and VMs, not the other workstations on my network.
pi-hole and bitwarden.
Do you actually host bitwarden yourself and rely on it? Your backup scheme must be extremely robust, I'd be too afraid to host it myself because of the possibility of losing everything.
Deleted my comment so I can provide some context.
Backup methods
Backing up apps
App links
Edit
Doh, forgot about how I actually back them up. It is much simpler than you think, and once you have the process in place (all automated) it takes roughly 10 minutes weekly.
Take all of those files, ZIP / password protect, dump in Cryptomator and put somewhere where your sync / backup solution will grab it. Easy peasy, 1-2-3.
[deleted]
I haven't gotten around to setting it up yet, but I just heard about Apache Guacamole and it seems really cool. Remote desktop gateway that works through a browser!
Definitely do it! If you're running docker there's an all in one image on docker hub so other than mounting volumes and getting port 8080 forwarded it's a one line install. I just set it up this weekend.
Try some configuration management. I'm running Salt and I love it.
that is a deeeeeeeeep rabbit hole. good luck, fellow salt friend!
Nice! I'm deep into the Puppet well myself.
[deleted]
Resilio Sync to keep my music library up to date on an old HTC M8 I use as a music player
Also, Atlassian's Confluence for an internal knowledgebase and documentation hub. Great for both homelabs and business use.