I have a robust monitoring system for my Jellyfin server, been running for the last few years. Checks in periodically, at least once per 24 hours and notifies me if it's down. Doesn't use any electricity, but does consume a good amount of Cheerios and mac & cheese.
Selfhosted
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
-
Be civil.
-
No spam.
-
Posts are to be related to self-hosting.
-
Don't duplicate the full text of your blog or readme if you're providing a link.
-
Submission headline should match the article title.
-
No trolling.
-
Promotion posts require active participation, with an account that is at least 30 days old. F/LOSS without a paywall has exceptions, with requirements. See the rules link for details. Tags [CBH] or [AIP] are required, see the links in Rule 8 for details.
-
AI-related discussions and AI-involved promotional posts have additional requirements for tagging, as noted in Rule 7 and the AI & Promotional Post Expanded Rules post, and find example disclosures here.
Resources:
- selfh.st Newsletter and index of selfhosted software and apps
- awesome-selfhosted software
- awesome-sysadmin resources
- Self-Hosted Podcast from Jupiter Broadcasting
Any issues on the community? Report it using the report flag.
Questions? DM the mods!
Does this monitoring system also happen to be a dependent on your taxes?
I've got a few of those services running.
Lies! It uses electricity!
Indirectly. But as another commenter pointed out, it balances out with the tax rebates.
You guys are monitoring your servers?
"can't access this thing, server must be down"
I'll fix it when I get home
I plan to perhaps, just maybe, I am not sure, to try self-hosting an email server. But I'll need a good uptime.
Although I am really scared about security.
Well, it’s a good learning experience. But….. don’t. Good luck getting around the major email providers’ smart screen shit. It’s probably not gonna happen. I tried a few years ago, and even microslop couldn’t figure out why their shit screen crap wouldn’t allow my emails. It’s not in there best interest to fix it. So they don’t. But it’s worth doing it in the short term to learn how to do it. Teaches you about email, spf, dkim etc. which actually does translate to real world skills. I used what I learnt setting it up in my real life job.
Yeah, email is one of the more annoying things to self host. Google rejects mine because my ISP has all their non static IPs on block lists (even though my IP hasn't changed once since I've been with them it's still not "static", can't change the reverse DNS) and I'm not paying 150% more a month for a business account to get a static IP. I ended up just using a free email relay. Still had to setup SPF, DKIM, DMARC, and all that jazz still.
I self-host my own email, have done so for years. I've not gotten pwned. Use an SSH key and a firewall, only expose the ports that should be public, have reasonable access controls, etc, just practise general cybersecurity—it applies to servers too.
I know everyone complains about hosting email. But I've not run into major problems. My emails get through spam filters and I have good uptime. It's also a fun learning experience.
I think that you have to do quite a bit more setup on email than most other services. There's SPF and DKIM and DMARC and so on. Plus spam filtering, setting up accounts and authentication, ...
Plus email is arguably more important than anything else, since it's kind of the lowest common denominator of the Internet; your dentist will probably send you an email to remind you of appointments, job offers will arrive via email, password resets arrive via email, and so on.
So it is a bit scary.
Having said that, as long as you're not hosting from a residential IP addresses it is definitely achievable. You can start with one domain and use it for low-value purposes (or for accounts that allow multiple email addresses, like GitHub).
Been running one for many years, mostly issue-free. Had to request whitelists way in the beginning, but things turned out ok.
Check out this super great guide. https://workaround.org/ispmail-trixie
Lights are on, server is on.
I recently found out that this is not always true.
I never said anything about it responding.
Whining from end users.
I notice something stopped working, or someone in the household notices.
Then I go fix it.
And the best time to notice that the backups weren't working is when you need access to the backup. 😅
Backups? In this economy?
But I need to get something working. I am about to move to VaultWarden and that makes me very nervous.
That's the neat part, I don't.
I pay a group of schoolchildren to refresh a group of browser tabs and yell if anything isn't responding.
There are plenty of previous threads with recent answers if you search.
Well I used to use VGA for my monitors, now its mostly displayport.
Seriously though, prom+Loki+alloy+a few other things to Grafana, with alerting in grafana.
Uptime Kuma! For HDDs I have them report their “ping” as percentage full
Can you teach me this wizardry?
Sure! In Uptime Kuma, add a monitor with type Push. It gives you a URL with a unique token, and the monitor goes into “down” state if nothing calls that URL within the heartbeat interval. Then have a cron (or a systemd timer or whatever) run this on the machine with the drive every 5 or 10 minutes:
use=$(df --output=pcent /mnt/yourdrive | tail -1 | tr -dc ‘0-9’)
curl -fsS -G “http://your-kuma:3001/api/push/TOKEN --data-urlencode “status=up” --data-urlencode “msg=${use}% used” --data-urlencode “ping=${use}”
Ping is of course meant to be response time, but Kuma will graph any number you toss in, so you get a “filled” chart for the drive. I also send status=down once usage passes a threshold. And if the box itself should fall over for some reason, the missing check-ins trigger the alert too.
(I typed this quick on my phone, so please double-check and probably don’t blindly copy-paste)
Use service, wait for 404 or data loss...
Cockpit. Total gamechanger for me. I don't have to ssh to my boxes individually to see what's going on. Instead it's a web UI AND you can connect basically every server you want to one instance.
https://github.com/cockpit-project/cockpit
That said. For individual servers?
htop/nmon (Get nmon straight from the source! https://nmon.sourceforge.io/pmwiki.php) for a birds-eye sysmon view of things
ps -ef | grep <whatever you're looking for> for investigating stuck/hung/zombie processes
dmseg
tail -f /var/log (or whatever log file you care about)
netstat for net statistics (i think ntop used to be a thing but idk anymore>...)
df -h to look at disk usage
those are all that come to mind right away.
EDIT: formatting and quick explanation of what each tool does
I use naemon (nagios fork) and grafana Loki Prometheus influxdb
I have the best detector for issues with my self hosted services: my wife!
Thats cool! My wife thinks i'm crazy just because i homeserve ...
It's running the private DNS for my phone. If it goes down, I realise quite quickly.
They're in my attached garage. In the living room when it's quiet I can hear them humming. Does that count?
I get a text pretty quickly.
I use nagios
I usually just walk down the hall and move the mouse. I don't really need remote monitoring. 😅
I use Alloy to collect Metrics of the host (Disk usage, CPU, Ram, etc.) and different logs. With Grafana the data is then displayed as a daschboard. If everything goes south, Alertmanager sends Mails to me
Zabbix
Prometheus + grafana. It's overkill tbh, most of the services restart automatically and I mostly ignore it. (it's an artifact from life where I cared about it)
My primary monitoring system is Beszel, but since I just migrated from VMs to LXCs it can’t properly see the memory/CPU usage, so it just checks for disk usage and liveliness. Home assistant with Proxmox & Dockhand integrations gives me the rest of my monitoring needs.
Uptime Kuma, only really care if it's up or not. Got about 30 services monitored, sending notifications to Signal.
Ah cool! How do you get notifications sent to signal. I've always avoided because i didnt like emails of my home setup leaving home in clear
https://github.com/bbernhard/signal-cli-rest-api
Its very finicky to setup, but once you get it working its pretty good. Sometimes needs to be reauthenticated though.
Kuma notification settings:

Looks like i don't have to expose ports or anything like that, right?
Nope, your uptime kuma speaks to it directly, and it basically acts as a signal client for you.
Edit: ah, i see! It's kuma that allows you to i guess by adding new device on phone Signal