My Home Server setup & looking for advice going forward
Posted Jul 25, 2023, 3:14 PM UTC
128
what did you use to draw this?
1
1
Really good diagram ☺️
I've been looking into getting either a minipc with the last gen AMD APU (with usb4 which fits your external GPU requirement) or to build my own thing using the Jonsbo N2.
A few resoruces for minipc
Personally I will go with the DIY Jansbo build since I intend to passively cool it and that case should allow me to put a big heat sink.
About your your diagram, I'm missing a bit more monitoring, even if you only run one server imho it is important to know what is and was the state of it so in case of failure you can diagnose it and react to the failure promptly.
For your case I would probably recommend netdata mostly because when I used it (a few years ago) it was excellent for a single server but its multi-server story had a few shortcomings (this might have changed). For a multi server solution I really like prometheus/victoria metrics + grafana, also grafana is really versatile and you might want to use it regardless and replace prometheus with some other time series database.
For logs, I have been using loki for a long time (with grafana as the UI), it is low maintenance and the docker driver has worked flawless for me.
One last note, I usually like to have my media files also available via smb, I don't see it in your diagram and I find it quite handy.
As an anecdote I have been managing a few deployments of prometheus/grafana/loki using their :latest docker images and updating them from time to time. It has never given us any issue, not once.
Regarding backups, you might want to also take a look at restic, there are plenty of comparatives for borg vs restic vs duplicati vs kopia.
1
Sidebar: Something I keep seeing is an abundance of selfhosters with sonarr and radarr, etc., and Plex....OMG are people Plex happy! How is no one getting busted for downloading stuff anymore? I was under the impression that it became much easier and more a priority to stop downloading a number of years ago. Is that just not a concern anymore? I remember getting a letter from my ISP back in like 2013 for downloading ONE episode of Boardwalk Empire that I missed on HBO. What gives?
1
How is no one getting busted for downloading stuff anymore
Usenet. Or if you must torrent, a VPN.
6
Streaming became available, turns out people would pay for streaming services and downloading became less of a problem - I guess the need to track down downloaders did too. However it does depend a lot on the country you're in, sometimes the privacy laws prevent whatever copyright organization sues people in your country from getting your personal data based on IP-address. Then the internet provider acts as the middle-man, but (again depending on country) can't really do anything about it.
You can also just use a VPN (or a proxy, but that's less secure), so they won't be able to track you down (easily).
1
I mean personally the country I'm in literally does not give a shit. Depending on how you pirate it might even be legally protected.
1
Aside from VPNs, there are a lot of private torrent sites out there with torrents that can't be easily monitored by the MPAA/whoever.
1
VPN connections for the server running Sonarr/Radarr/Torrent. Many people also use private USENET/Torrent providers that do not log.
1
This is an inspiration, I have been wanting to draw a similar diagram but procrastinated. Can you please share your excalidraw source file?
1
Hello Fellow Belgian?
1
B-box gave it away? 😅
1
This is a great diagram. The style reminds me of the ups commercials from a few years ago (although those were all brown "ink" on a white background).
All this runs on a laptop?
1
Yeah, it runs on an Asus G750J (i7 4700HQ, 16GB RAM)
1
Impressive! Way to go!
1
Hello! The only advice I can give you is: With using Jellyfin with Google TV (if you're talking about a Chromecast with a remote control), sideload Kodi on it and then install Jellycon addon in Kodi. It just works like a charm!
1
Hey can you please share how you configured this for your certs? It seems like you are only exposing this via tailscale. Also what’re you doing on the ACL and security side of things?
1
SSL Certificates are managed by Traefik through DNS Challenge (Cloudflare in my case)
The ACL now is set up with tags, of which one is the tag:server, one is tag:github-actions, and the others are tag:user.
I've set up the ACL so that tag:github-actions can only access tag:server, users can, for now, access everything.
"acls": [
{"action": "accept", "src": ["tag:user"], "dst": ["*:*"]},
{"action": "accept", "src": ["tag:github-actions"], "dst": ["tag:server:*"]},
],
It isn't really needed, but it's an extra security measure and I wanted to try out the ACL. I want to go a bit further with it in the future.
1
Baguette 🇫🇷
1
You really need an outgoing proxy now ^_^
Same blacklist you have on AdGuard can be used for squid.
Squid will cache your packages then all updates will be faster and all requests to the evil world safer.
Of course Squid /etc/resolv.conf can be pointed to AdGuard.
Have a nice squid day :)
1
Have been trying this today and got this set up in docker, at least the requests go through, but getting all kinds of denied errors and can't really find a config that just allows everything and logs requests, so I can fine-tune it later. May try again later but I've seen people use nginx as forward proxy, I'm much more familiar with nginx config files so might give that a go too.
1
For logging and more you could self-host with the grafana suite.
- logging ( loki and promtail )
- metrics ( prometheus )
- traces ( tempo and grafana-agent )
- dashboards ( grafana )
And there are a lot of premade dashboard for grafana, checkout https://grafana.com/grafana/dashboards/
1
Got a grafana setup ready, I'll check out those dashboards, thanks!
1
Wow, that's all really cool, I hope to get something like this together myself as well someday. As I'm a newbie I unfortunately cannot offer any tips, however I would to know: how did you even start all this?
The most I have currently is a raspberry pi running locally with docker servives of my apps, that I connect eith using discord bots that I made (this is because I am quite privacy focused but I don't have the knowledge of securing my ports to expose them to the internet and whatnot).
So, do you have any good resources that you used or know of, to get started with building such an ecosystem?
1
I went a different route with AdGuard and set up a small Linode to host it in the cloud and updated the DNS on my router, so all DHCP assigned traffic goes through it. I configure servers using static and can send them to AdGuard or set up Google DNS. Using this method, I can also update other family/friends routers to go through the same filters. Better than on prem for me. I found someone in the household had installed a bunch of crap on their computer and approx 1/3 of the DNS lookups were trying to reach an FTP site for webcam uploads.
1
Bonjour voisin, beau diagramme !
Have you implemented all this because you know the subject or have you followed any guides/resources?
I'd like to set up something similar but I clearly don't know how, so if you ever have any links to share, I'd love to read them.
Also, for Tailscale, is it necessary for people connecting to your domain to have a client installed or is it publicly accessible?
1
I haven't followed any guides, but I'm familiar with docker and servers in general (I'm in web development). It's mostly trial and error though, I've swapped and change the reverse proxy and DNS stuff several times already (used nginx, caddy, dnsmasq, etc before).
I'm mostly learning ad hoc, if I want/need something, I just research and try and get familiar with it. So I can't really give you links, as I've been reading pretty much all over the place, there's no single guide I've followed.
To answer the accessibility part, I'm currently the only one accessing it and all my devices (even my tv) is connected to Tailscale.
If I wanted another user to access my things, I would either:
- Add them as a user in Tailscale, have the install the client and configure ACL's
- Set whatever they need access to public using Cloudflare Zero Trust tunnels and create accounts for them (either on the app itself or through Authentik)
1
I'd throw in a sandbox to detonate all suspicious files.
1
What did you use to make that diagram?
1
Hardware
Everything is running off of a 9 year old gaming laptop which has it's benefits (battery, keyboard & screen when stuff goes wrong), everything is stored on external drives.
Because the battery has died and the replacement is about 160 EUR, and the desire to run AI models locally (LLM's + text-to-image) , I've been looking to replace the laptop completely. Suggestions always welcome, but not looking to invest thousands in it (leaning towards something that I can later get an external GPU for).
Software
I've tried a lot of things, the following is the list that has stuck around mostly.
op run(See https://developer.1password.com/docs/cli/reference/commands/run/)Management
I currently manage this manually through command line, all config in stored in a Github repository. I started without a plan and improved along the lines, have been thinking about using Ansible but it's rusty to say the least and I've never managed docker via Ansible.
I can start/stop/restart/update containers through Github Actions, though I mostly just SSH.
I'm personally not a fan of UI based setups like Proxmox/Portainer/etc
Next steps
Backups: Since recently I no longer have a backup system, have been backup up to an old drive that gave in. So I'm currently looking to back up:
For that, I'm looking at:
Logging: I've been wanting to centralize logs as to not be SSH'ing into the server and manually digging through them anymore. So I'm looking for something:
Really good diagram ☺️
I've been looking into getting either a minipc with the last gen AMD APU (with usb4 which fits your external GPU requirement) or to build my own thing using the Jonsbo N2.
A few resoruces for minipc
Personally I will go with the DIY Jansbo build since I intend to passively cool it and that case should allow me to put a big heat sink.
About your your diagram, I'm missing a bit more monitoring, even if you only run one server imho it is important to know what is and was the state of it so in case of failure you can diagnose it and react to the failure promptly.
For your case I would probably recommend netdata mostly because when I used it (a few years ago) it was excellent for a single server but its multi-server story had a few shortcomings (this might have changed). For a multi server solution I really like prometheus/victoria metrics + grafana, also grafana is really versatile and you might want to use it regardless and replace prometheus with some other time series database.
For logs, I have been using loki for a long time (with grafana as the UI), it is low maintenance and the docker driver has worked flawless for me.
One last note, I usually like to have my media files also available via smb, I don't see it in your diagram and I find it quite handy.
As an anecdote I have been managing a few deployments of prometheus/grafana/loki using their :latest docker images and updating them from time to time. It has never given us any issue, not once.
Regarding backups, you might want to also take a look at restic, there are plenty of comparatives for borg vs restic vs duplicati vs kopia.
Thanks a lot! That's a lot of useful info to go through.
I actually have a Grafana+loki+promtail setup that I was testing already that I completely forgot about (stopped the containers), I'll check the docker driver and will give netdata a shot too!
Regarding smb, I haven't really needed it yet, that reminds me I have DLNA through Jellyfin and should probably include that in the diagram.
I'll add restic and Kopia to the list to check out! Thanks!
I currently use ansible and have been bitten multiple times by python and pip modules not working correctly. I’m slowly moving to HashiCorp Nomad as orchestrator but still uses docker internally.
For those using Synology NAS I have a package for it at https://github.com/prabirshrestha/synology-nomad.
I recently submitted patch for ssh support for nomad gitops so everything can be deployed via git ops and I don’t need to use ansible except for the first bootstrap which is installing nomad and creating users. https://github.com/jonasvinther/nomad-gitops-operator. Nomad now includes variable and service discovery so you don’t need consul or vault if you are starting.
As for logs there is https://openobserve.ai/. Rather than installing multiple tools you just install this one.
Ah yes, I've also been down the python pip module rabbit hole before with some other stuff.
Nomad looks interesting to try out, Vault also seems to have 1Password integration so that might all fit together better