Senpai-R710, 2x X5675's, 32GB RAM•

New to the gang, been camping this sub like crazy and learning a lot as I go.

R710 - 2x X5675's, 32GB RAM

  • 2x 300GB 15k's (Raid 1)
  • 4x 300GB 15k's (Raid 10)
  • ZFS Mirror WD Blues (1TB)

Running Proxmox 5.0:

  • Time Machine Server
  • Debian backup Server (to off site)
  • Plex server
  • Nextcloud server
  • Some test VM's for testing random software before I implement it.

Bananapi M1 // Ubuntu 16.04 Minimal running:

  • Reverse proxy
  • OpenVPN for home access
  • IPMI scripts for powering on the R710 (I don't run it 24/7 ATM)

Decided to run SMB on the host itself for giving network shares, even though I wanted a NAS VM. Can't complain though, ZFS has been really easy to setup together with SMB and it's respective groups.

18
SkepticNerdGuy•

So just yesterday I set up SMB via a turnkeylinux fileserver container. I have proxmox running a ZFS storage cluster and it has stores for images and VMs as well as one for what is supposed to be the SAMBA share. I passed the mount to the turnkey filse server as a bind mount. So far so good, but I have not done thurough testing just yet.

3
devianteng•

What are you running for your Time Machine Server? I mean, I have an OSX KVM instance in my PVE cluster, but the only reason is for the server package to run as a Time Machine server. Other simpler solutions are always nice.

3
Senpai-R710, 2x X5675's, 32GB RAM•

Ubuntu 16.04.3 LTS VM with Nettalk, and Avahi.

Took quite a while to learn how to set it up and it can be a real pain to fix it breaks (I usually don't touch it after it's setup and working).

But I keep the timemachine data drive separate from the Ubuntu drive, so if anything breaks I can just restore the Ubuntu server to a working state without losing my timemachine data.

Works perfectly though! May I ask, how did you virtualise macOS (do you have a guide for this)?

1
devianteng•

Ah, yes. I've played with Avahi before but couldn't get it working quite right.

There are several guides out there for running OSX on Proxmox, but I've never had full success directly following one of those guides. But, with enough patiences, you can piece things together. Here is my qemu file, which should help:

args: -smbios type=2 -cpu Penryn,kvm=on,vendor=GenuineIntel,+invtsc,vmware-cpuid-freq=on -kernel /mnt/pve/nfs_mjolnir/template/qemu/enoch_rev2889_boot
balloon: 0
bootdisk: ide0
cores: 8
cpu: host
ide0: ceph_vm:vm-137-disk-1,size=40G
machine: pc-q35-2.4
memory: 8192
name: osxserver
net0: e1000-82545em=6E:8E:DE:79:D0:F9,bridge=vmbr1
numa: 0
ostype: other
sata1: nfs_mjolnir:137/vm-137-disk-1.raw,backup=0,size=500
scsihw: virtio-scsi-pci
smbios1: uuid=e446eb44-6d53-4f86-ad32-427e7a6de404
sockets: 1
vga: std  

As you can see, I have a OSX QEMU instance with 8 vCPU cores and 8GB RAM, with a 40GB disk from my Ceph pool. I've also got a 500G drive mapped via NFS, that is used only for Time Machine backups. For the most part, it's been rock solid.

2
ethanobcom•

Old core 2 duo laptop running kvm with a few CentOS VMs, planned on upgrading to a real server but don't have the money ATM

10
nail_nail•

The good thing of old hardware is that you get really motivated at juicing every single horsepower out of it.

12
soothsayer011•

Same

1
AdjustableCynic•

What're you using the VM's for, out of curiosity?

1
ethanobcom•

Just a few Webservers and things, also a few windows VMs and some windows server VMs.

2
MittensGBNDL380 G6 | Major Mittens#2794•

DL380 with Proxmox, running an Arma server and Insurgency.

Planning on buying more HDDs and possibly setting up plex. I'd like to dabble in a little Pfsense and tempted to pick up an R210ii.

If my phone wasn't potato and I wasn't afraid of the hatred I'd get for showing off the shockingly bad mounting solution, I'd show off my server that's strapped to a wall.

7
[deleted]•

[deleted]

1
FooHentai•

DL380

Generation?

If my phone wasn't potato and I wasn't afraid of the hatred I'd get for showing off the shockingly bad mounting solution, I'd show off my server that's strapped to a wall.

Naw c'mon, we'll be kind. Let us laugh with you, only slightly at you.

1
gscjj•

My lab is finally in a usable state now that I've cleaned out some old hardware, spent time designing it correctly and bought 6 1TB 7.2K(2.5) Drives. I'm no longer cramming everything into a couple 136GB disks.

All running on a Dell R420, 48GB RAM, 3TB Raid10 and 120GB SSD:

  • dev01 - CentOS7 Box to mess around with
  • log01 - Graylog2
  • plex01 - plex
  • plex02 - PlexPY & PlexRequest
  • dc01 - W2016 Core AD DC, DNS
  • dc02 - W2016 Core AD DC, DNS, DHCP
  • file01 - W2016 Core File Server
  • wds01 - WDS & MDT
  • prox01 - Nginx Reverse Proxy
  • rds-br01 - RDS Broker
  • rds-gw01 - RDS Gateway & WebAccess(RemoteApp)
  • rds-sh01 - RDS Session Host (my "jump box")
  • ind01 - Jackett
  • lib01 - LazyLibrian
  • mov01 - Sonarr
  • tv01 - Radarr
  • tor01 - Transmission
  • ans01 - Ansible
  • pf01 - pfsense for lab
  • psc01 - vSphere PSC
  • vcsa01 - vCSA

Plans for the future?

  • Adding more host - I spotted some cheap R710s locally I might pickup
  • Physical pfSense - looking to move pfSense out of the lab so I can start messing with NSX
  • RAM, and more RAM
7
_Noah271•

How do you license RDS?

3
crazyonerobDevOps Engineer and breaker of the rack•

MSDN is how I do it.

3
_Noah271•

I need to do that.

1
Jermny•

Bizspark is the best way to get a free multi-year msdn license.

1
karelvanheckeR710 | ESXi | FreeNAS | pfSense•

How much disk space do you allocate to the sonarr, radarr and transmission vm's? Also, do you route their traffic through a vpn or not at all?

1
gscjj•

They all have 16GB, everything is downloaded to the file server so they don't need much space.

Everything except AD and SSH go through a VPN

1
Irravian•

Current Setup:

R610

  • 2x X5570

  • 96GB RAM

  • 5x 10k 146GB, 1x 500GB SSD

  • Hyper-V hosting PFSense, AD, Couch/SickRage/Deluge, Plex, NextCloud,MediaWiki, Grafana, NGinx reverse proxy, and game servers.

Whitebox

  • 24 3.5" Bay case of unknown manufacture (craigslist)

  • i3 7100

  • 32GB RAM

  • 4x 3TB WD Green (Raid 5) Primary array

  • 3x 250GB WD Blue (Raid 0) Scratch space

  • FreeNAS box

R410

  • 1x E5630

  • 8GB RAM

  • 3x 1TB HD

  • Currently unused, will likely get a small boot drive and become Windows Storage Spaces to play with Clustered Hyper-V (since FreeNAS, afaict, does not support whatever Windows needs from the SMB share for that).

Recently acquired:

R610

  • 2x X5650

  • 12GB RAM

  • Will likely be migrating VMs here, pulling the RAM and NIC from the existing 610

R610

  • 2x 5570

  • 12GB RAM

  • Will likely be setting this up with the existing R610 (once the VMs are migrated) in a cluster for studying.

R610

  • 2x x5550

  • 12GB RAM

  • Not yet sure what this will be doing. Potentially adding a secondary AD VM or PfSense CARP

R710

  • 2x E5520

  • 2x73GB 4x146 10k

  • 12GB RAM

  • May possibly be an ESXi or Proxmox box with a graphics card to try something other than Hyper-V. May end up as UnRAID, the aforementioned Windows Storage Spaces array, another Hyper-V box, or some other weird thing.

Planned Tasks:

  • Get WSUS up

  • Get Ombi up

  • Get GitLab up

  • Rework and rewire networking (move VMs to internal switches and do Link Aggregation on the physical NICs)

  • Collapse the number of PfSense interfaces I'm running

  • Get some SQL servers and IIS boxes back up for testing things for my .Net development day job

  • Figure out better bandwidth monitoring on PfSense or replace it

  • Consolidate my backup strategy

  • Resist the urge to buy lots and lots of RAM just yet

  • Stop ordering servers

6
Haribo112•

Stop ordering servers

HERESY!!

15
pdhcentral•

I wish I was still like you... living the dream.

3
derasdn•

Currently: Nothing actively.

Got the first week off after I moved into my new apartment, so Im planning on actually doing my homenetwork this week.

Still trying to decide on a sofware router to use. Anybody got suggestions that arent pfsense?

5
syncopsta•

Vyos and Mikrotik would be a choice if you search for a router and maybe a basic firewall without layer 7 features.

3
zumtest99•

I am using Sophos UTM v9 with free Home License running as a VM.

2
derasdn•

Smallish update. My addiction has taken on legs and is running faster than Usain Bolt. I bought 3 servers over the last 2 weeks and I got gifted another one today together with my purchase of a KVM console and KVM over IP Switch... (cause why not go full nuclear if youre in the hole already)

Sneakpeek of the front side with everything in it.

Originally I was planning on enlarging the door to fit my old HP 42U rack inside, but it turns out that its more expensive than getting a new rack that comes as a flatpack and I can put together in the room itself.

Now I have to do the cabling. Cant take verything online yet anyway. Electrician still has to install separate fuses for my office and the server room.

1
Fewleftstanding•

Were you trying to post a photo? I don't think it worked.

1
derasdn•

Shows up for me. Made the link larger.

Here it is again: link

1
Alekoy•

Mainrig (for browsing, gaming, 3D and editing)

  • Xeon E5-2699v3 (all cores turbo microcode)
  • 32GB ram
  • GTX980ti
  • Custom watercooling loop.
  • 10gb lan to fileserver
  • 4U case

CCTV (ubiquiti)

  • ASRock N3700-ITX
  • 16GB ram
  • 4TB Skyhawk hdd
  • 1U case

ESXi

  • Dell Precision R5500
  • 2x Xeon X5690
  • 96GB ram
  • 10gb lan to fileserver
  • 250gb 850evo datastore1
  • 4x 2tb hdd datastore2

Fileserver

Currently rebuilding my fileserver, have ordered but not recieved the following for use with my old Supermicro 846 (sata backplane) case.

  • Dell R610, E5520, 16gb ram.
  • Dell H200E sas controller
  • Supermicro CSE-PTJBOD-CB2
  • 6x SAS to SATA cables with SGPIO
  • rails, trays and cover for the R610
  • 2x SAS 8088 cables

Need to order more rams for the 610

5
Alekoy•

well.. I was looking on ebay for more ram for the 610 (arrives tomorrow)

And I "accidentaly" ordered two R410 servers instead...

So now I need to order even more ram and some more rails.

3
[deleted]•

[deleted]

1
Kamma77•

Just replaced my ML350 G6 and DL360 G7 with two identical DL380 G7's with the following configuration:

  • 2 x Intel Xeon E5649 Six Core @ 2.53 GHZ
  • 60GB RAM
  • 2 x 300GB HP SAS 6G DP 10K (RAID1+0)
  • 2 x HP SC08e 6Gb SAS HBA
  • 2 x 460W PSU

I'm also in the process of replacing my current physical DC (a refurbished WatchGuard QMS 500 - basically a Supermicro board with a Celeron, 4 gigs of RAM and a random 500GB SATA disk in a Supermicro 1U rack-mounted case painted red and 'WatchGuard' written on it) with a DL360 G7 (single quad-core Xeon and 12 gigs of RAM, dual PSUs)

I could have just kept the DL360 G7 I already had, but it was a bit too overkill for just a DC. This one is too, but I planned to use it as somewhat of a management server aswell. I also (for some reason) liked the fact that it is the 4 x 2.5" bay variation with a CD-drive. The difference in power draw between the two is definitely noticable, which was a factor in my decision, aswell.

What I have running right now is the following:

  • CS:GO Dedicated Server on Ubuntu 16.04
  • Domain Controller on Server 2012 R2
  • PRTG Server on Server 2012 R2
  • SQL Server 2016 on Server 2016
  • IIS Web Server on Server 2012 R2
  • Milestone XProtect Essentials Server on Server 2016 (haven't added any cameras yet)
  • Development Server on Server 2012 R2 (for a friend to play around with some SQL and Web development stuff)
  • Exchange Server 2016 on Server 2012 R2
  • Xibo Digital Signage on Server 2012 R2 (wanted to try and play around with it a little bit - we currently have several clients at work who are interested in that sort of thing)

What I have planned right now:

  • Cluster the two hosts - I'm still trying to figure out what I wanna do in terms of storage. Maybe fill both of the hosts up with 300GB disks and add them all to a storage pool? I was also considering using my old MicroServer with a couple of 2TB disks and use iSCSI, but I'm not sure. Suggestions welcome!
  • Setup a DPM server - I got a license and figured it would be a nice way to backup all of the VMs, etc.
  • Finish setting up the Exchange Server - I f'ed something up during the setup, so it doesn't work 100%. I'm yet to figure out what I did wrong and correct it, so I can actually use it.

EDIT:

  • I forgot to mention that I have yet to segment my network properly with VLANs.
5
Hovertac•

What are you currently running?

Asus RT-N16 as Wireless AP

TP-Link Unmanaged Gigabit Switch

HP Proliant DL360 G7 2x E5645 (6-Core) 64GB RAM 8x 73GB 15k (RAID 5)

  • pfSense
  • Pi-Hole
  • Plex
  • Sonarr/Radarr Torrent Box w/ VPN
  • Domain Controller
  • Apache/MySQL Server
  • Minecraft Server
  • Guacamole
  • Windows 10 Client
  • Mac OSX Client

FreeNAS (5x 2TB RAID 5, 6GB RAM) storage for Plex and various other things.

What are you planning to deploy in the near future?

So far this week I've ordered and have or am waiting on:

  • 4 HP ProCurve 2824 Gigabit Switches
  • 1 HP ProCurve 2600-8 PoE 10/100 Switch (For Cameras/APs)
  • 2 Cisco Aironet LAP1142N AP's
  • 4 2MP IP Cameras
  • 1000ft CAT6
  • Bunch of wall plates & keystones

Currently working on:

  • Physical Windows 10 box
  • DVR VM for cameras
  • IDS/IPS on pfSense
  • VLANs

Would like to have:

  • BACKUP!!!
  • OpenVPN
  • Exchange
  • If I get the time to play games again, various game servers would be nice
  • Any other suggestions appreciated, I work in help desk and looking to build my experience
4
lm26sk•

R210II -8GB Ram -500GB Sata drive -Running PfSense 2.4

DL360 Gen7 -32GB ECCRam -2x 146Gb 15k Drives -Proxmox 5 VE running NethServer , Debian9 container with Pi-Hole.

Gaming -i7-6700k Oc'd to 4.6ghz -32 GB DDR4 Kingston -240 GB SSD -1.5Tb HDA ( 500gb Win10,1Tb Storage) -RX480 8GB -NZXT Case + Cpu Liquid Cooling -Windows 10 -VirtualBox running Kali ,Ubuntu 17 + Cinnamon , Debian 9 + Cinnamon

Future

Cisco Switch -Have to look deeper which one to choose

PAtch panel - Have no clue for what since i got 4 cables total,but hey,why not?!

R710 -- Just because everyone has one ! No plans, but for sure Proxmox Node

Unify Network - love the way they look

2
brianbrifri•

The final pieces are enroute but here is what I am currently building!

Supermicro CSE-825TQ-R700LPB

  • x2 700 Redundant PSU
  • Supermicro MBD-X9DRI-F-O motherboard
  • x2 Xeon E5-2680 v2 CPUs
  • SAS 9211-8i Controller Card
  • 20TB HDD space

Will be using it for Plex, NAS, and linux VMs to develop on.

Super excited to get this thing going!

2
fishtacos123•

What are you currently running?

Network

LB6M 24 port 10GB SFP+ ethernet switch – modified with silent fans, runs hot as hell.

HP 1910-16g 16+4 SFP GBE switch

UniFi AC Lite Access Point

6x Mellanox CX-2 10GB EN cards - each host connects to 10GB switch, with fallback on 1GB switch.

 

Storage

vSphere VSAN for VM consumption – 1.8 TB SSD total, ½ of that usable in normal scenarios (mirrored)

Starwind VSAN for 2ndary VM consumption – 1TB mirrored iSCSI pool, SSD backed, need to experiment with NFS & SMB, supposedly supports both in free version.

~46TB HDD storage - for personal backups and Plex & other media collections

 

6 Compute nodes – all running ESXi 6.5u1

1x 4u SuperMicro case*

S2600CP2J motherboard

2xE5-2670 v1 @2.6ghz (16 cores)

128gb DDR3 1333mhz

1.1TB SSD storage for VSAN

 

1x White box tower case

S2600CP2J motherboard

2xE5-2670 v1 @2.6ghz (16 cores)

128gb DDR3 1333 Mhz

1.1 TB SSD storage for VSAN

 

2x R210 IIs

E3-2140 v2 @3.4ghz each (4 cores each)

24GB DDR3 1333mhz each

(considering adding some storage & creating standalone cluster with these… maybe not, maybe a different hypervisor that’s kinda still “shiny” like Acropolis)

 

1x ZTsystems 1U

2xE5-2620 v1 (16 cores)

32GB DDR3 non-ECC

(this one is a prime candidate for a 128GB RAM upgrade, provided I ever outgrow my current lab)

 

1x White box

3820 @3.6ghz (4 cores)

32GB DDR3 non-ECC

 

UPS Protection

APC UPS 450VA – older unit, still works, protects switches & AP

CyberPower CP15PFCLCD 1500VA – protects 4U node, which also runs important network & storage “infrastructure” VMs, along with mechanical storage.

 

The VMs (all protected by HA & complemented by DRS):

Storage Server VM – WinServer 2012R2, 3xLSI 2008 8-port SAS cards passed through to VM for direct access to the drives.

pfSense – router VM, provides internet for whole house, as well as reverse proxy services via HAProxy. Also provides gateway/firewall services for additional VLANs in the network as needed for various projects.

2x WinServer 2012R2 DCs – LDAP services for various projects. DNS & DHCP on VLAN4 currently :)

Torrents – WinServer 2012R2, uTorrent, downloads to share on main storage VM

OpenVPN – AS appliance

TOR relay node

Cloudron server Sandstorm server

Yunohost

2x Starwind VSAN nodes*

vCenter appliance

VSAN witness appliance

Steam streaming – Windows 10 with a passed through Quadro 2000 card

Plex server – WinServer 2012R2, serves media shares from the storage server via SMB shares.

Veeam & Vembu VM = Monitoring & Backup by Veeam, and easy scheduled backup running Vembu BDR,

GNS3 VM –

Windows 7 –

Cinnamon 18.2 –

Ubuntu 17.04 –

OSX Sierra –

10x mining VMs – mining Monero coins with spare CPU cycles.

A nested cluster of ESXi hosts - for an NSX learning project.

A nested cluster of Hyper-V hosts - for some Network Virtualization and SCCM learnings.

Various OS templates - for quick deployment, mostly Windows, one Ubuntu instance.

 

TODO:

Reinstall Blue Iris and my WiFi security cams.

Remote desktop functionality for easy access to lab – currently relying on Teamviewer and a Windows Vultr VPS for times I need sneaky remoting.

Automated media acquisition - Want to automate my media coughs errrhhm, acquisition with tools like CouchPotato, SanNZBD etc.

Setup log server

Setup dashboard/monitoring of some sort

2
apt-get_Debian 9 + QEMU/KVM•

Current: Intel Pentium G4500 (2) @ 3.5GHz with 4GB RAM and 1TB WD Blue. I also have a Cisco Catalyst 3560 48-TS, which I'm currently playing around with too. It runs...

  • Debian Stable (Stretch/9) as Host OS
  • NGINX Webserver w/ PHP-FPM
  • Tor Daemon, which is a frontend for the NGINX instance, creating a hidden service
  • MariaDB instance with some test databases
  • Private Git Repo for upstream projects
  • Netdata to monitor server, also routed through the tor daemon, creating a second hidden service
  • QEMU/KVM to run like two VMs at a time (I can spin up a VM over SSH if I'd like)

All with no more that 400MB of RAM being used idle. Future plans:

  • Implement a MediaWiki instance to work with PHP + MySQL/MariaDB
  • Get an actual rack server (R710)
  • Setting up Xen + SELinux
2
Kallukoras•

Only got a very small lab. But i like the small form factor and low noise because i only got a room in a shared flat.

Intel NUC I3 6100i
* 250GB m.2 SSD
* 120GB Sata SSD
* 32 GB Ram
Proxmox 4.4 (soon 5.0 update planned)
* Plex Server
* Monitoring Server (Nagios)
* Pihole
* Seafile Server
* Ethereum Node
* Docker
Qnap TS 251

  • 16 GB Ram
  • 2 3TB Raid 1
  • VPN
  • File Storage for Proxmox Backups,Plex (via NFS)
2
ohsheit•

Buying another pc. The old 2012 Macbook I was running my lab on decided to fall over, I had the lights off and stepped on it so the eth port is done. Might have a sweet deal on 2x Thinkcentre on CL!

2
chubbysumoJust turn UEFI off!•

probably gonna add a dedicated server in my loft for BlueIris and my IP cams. I am using my storage server for it right now, as a single e5620 works fine for 4 cams, and it has the storage space, but I will either move that upstairs, or get a quiter tower machine to hide in a space up there to keep hidden in the event of a theft.

2
[deleted]•

[deleted]

1
hawaiizachwww.creedthoughts.gov.www/creedthoughts•

Noob here, how are you gaming in a VM? I’m sure it’s a really stupid question, but wouldn’t the Remote Desktop lag be too much?

1
[deleted]•

[deleted]

1
Haribo112•

steam's lan fuckery

steam's lan fuckery ®

7
BradPatt•

has multiple outputs(hdmi, dp and vga). so hdmi to windows

I thought the only way to assign a graphic card was with all the output (of that graphic card). With what you did, can you give another port, let's say DVI to a different VM? (At the same time)

2
rewirx•

As far as I know, this is true. I'm assuming he's using integrated graphics for the host machine and is dedicating his video card to the Windows VM.

2
[deleted]•

[deleted]

1
[deleted]•

[deleted]

1
CSTutorRHCALAB (6 R610, 2 R510, 1 FS7500)•

My ceph1 and ceph2 nodes are now using all brand new 2 TB drives hugely increasing my storage capacity (and durability). Well... The boot drives are still using the old drives in RAID 1. I'm going to use all the removed drives as spares so it'll be a while before I buy new ones for that.

Now that I have enough capacity on ceph, my next goal is to move the content for Plex from my WDMyCloud to a ceph RBD disk.

I also want to move personal documents from WDMyCloud to a RBD disk.

I want to use WDMyCloud exclusively for onsite Proxmox backups and TimeMachine.

1
finish06•

That is cool you are using ceph for bulk storage. What is your base OS?

1
CSTutorRHCALAB (6 R610, 2 R510, 1 FS7500)•

CentOS 7 for the ceph nodes.

1
notrufus•

Have an i5 7400 with 16gb of ram and 2 2tb reds. Running proxmox. List of VMs: openmediavault that's also a docker host for Jackett, Unifi, and Organizr. Also has plugins for sonarr, radarr, Plex, deluge, and openvpn. Pi-hole Docuwiki And one VM for caddy that runs a SSL reverse proxy so I can access all of these from wherever I want and let friends use them as well.

1
_Ritual•

Currently

At home:

Raspberry Pi 3

4tb WD NAS

In the cloud:

A couple of VPS that I use for hosting small personal websites and a Minecraft server.

(Gotta start somewhere, right?)

Soon

I'll be upgrading and want to spend the money to get something that will last me a few years, and allow me to spin up all the VMs I desire for tinkering with. Including but not limited to - pfSense, web servers, game servers, media servers and something to allow me to better learn Docker. I also want to upgrade my HomeAssistant stuff running on the Pi to something more reliable and less slow.

Found a few options on eBay including this thing:

Proliant DL380 G6

2x X5670 @ 2.93GHz

72GB RAM (18 x 4GB PC3-10600R)

2x 300GB 2.5" 10k SAS

P410i 512MB

ILO2

4x HP NC3821 GB LAN

I just don't know why it's listed for the price it is (under £200) and if its worth it, or if I should spend the extra £100 and get a R710 with less spec but seemingly more support.

Research continues, so hopefully it doesn't sell before I decide.

1