Sorry to hear that. Do you know how it was compromised and how you found out?
linuxmemes
Hint: :q!
Sister communities:
Community rules (click to expand)
1. Follow the site-wide rules
- Instance-wide TOS: https://legal.lemmy.world/tos/
- Lemmy code of conduct: https://join-lemmy.org/docs/code_of_conduct.html
2. Be civil
- Understand the difference between a joke and an insult.
- Do not harrass or attack users for any reason. This includes using blanket terms, like "every user of thing".
- Don't get baited into back-and-forth insults. We are not animals.
- Leave remarks of "peasantry" to the PCMR community. If you dislike an OS/service/application, attack the thing you dislike, not the individuals who use it. Some people may not have a choice.
- Bigotry of any kind will not be tolerated. This is an LGBTQ+-friendly community -- if that is a problem for you, you should leave.
3. Post Linux-related content
- Including Unix and BSD.
- Non-Linux content is acceptable as long as it makes a reference to Linux. For example, the poorly made mockery of
sudoin Windows. - No porn, no politics, no trolling or ragebaiting.
- Don't come looking for advice, this is not the right community.
4. No recent reposts
- Everybody uses Arch btw, can't quit Vim, <loves / tolerates / hates> systemd, and wants to interject for a moment. You can stop now.
5. π¬π§ Language/ΡΠ·ΡΠΊ/Sprache
- This is primarily an English-speaking community. π¬π§π¦πΊπΊπΈ
- Comments written in other languages are allowed.
- The substance of a post should be comprehensible for people who only speak English.
- Titles and post bodies written in other languages will be allowed, but only as long as the above rule is observed.
6. (NEW!) Regarding public figures
We all have our opinions, and certain public figures can be divisive. Keep in mind that this is a community for memes and light-hearted fun, not for airing grievances or leveling accusations. - Keep discussions polite and free of disparagement.
- We are never in possession of all of the facts. Defamatory comments will not be tolerated.
- Discussions that get too heated will be locked and offending comments removed. Β
Please report posts and comments that break these rules!
Important: never execute code or follow advice that you don't understand or can't verify, especially here. The word of the day is credibility. This is a meme community -- even the most helpful comments might just be shitposts that can damage your system. Be aware, be smart, don't remove France.
At least you had a backup

I've been quite stupid with this but never really had issues. Ever since I changed the open ssh port from 22 to something else, my server is basically ignored by botnets. These days I obviously also have some other tricks like fail2ban, but it was funny how effective that was.
Almost the same here. I also change some ssh settings: disable root login, disable password, allow only public key login. That's about it. I never had any problems.
I do worry about putting up public servers that other people might rely on because there's something I might not realize making it vulnerable.
So far I have pubkey root login only on the VPSs I'm messing around with, but my ol' reliable private key from 6 years ago might be beginning to fall behind on encryption standards.
One time, I didnβt realize I had allowed all users to log in via ssh, and I had a user βsteamβ whose password was just βsteamβ.
βHey, why is this Valheim server running like shit?β
βWtf is xrx?β
βOh, it looks like itβs mining crypto. Cool. Welp, gotta nuke this whole box now.β
So anyway, now I use NixOS.
Good point about a default deny approach to users and ssh, so random services don't add insecure logins.
I'm confused. I never disable root user and never got hacked.
Is the issue that the app is coded in a shitty way maybe ?
You can't really disable the root user. You can make it so they can't login remotely, which is highly suggested.
sudo passwd -l root
This disables the root user
There's no real advantage to disable the root user, and I really don't recommend it. You can disable SSH root login, and as long as you ensure root has a secure password that's different than your own account your system is just as safe with the added advantage of having the root account incase something happens.
Basic setup for me is scripted on a new system. In regards to ssh, I make sure:
- Root account is disabled, sudo only
- ssh only by keys
- sshd blocks all users but a few, via AllowUsers
- All 'default usernames' are removed, like ec2-user or ubuntu for AWS ec2 systems
- The default ssh port moved if ssh has to be exposed to the Internet. No, this doesn't make it "more secure" but damn, it reduces the script denials in my system logs, fight me.
- Services are only allowed connections by an allow list of IPs or subnets. Internal, when possible.
My systems are not "unhackable" but not low-hanging fruit, either. I assume everything I have out there can be hacked by someone SUPER determined, and have a vector of protection to mitigate backwash in case they gain full access.
- The default ssh port moved if ssh has to be exposed to the Internet. No, this doesn't make it "more secure" but damn, it reduces the script denials in my system logs, fight me.
Gosh I get unreasonably frustrated when someone says yeah but that's just security through obscurity. Like yeah, we all know what nmap is, a persistent threat will just look at all 65535 and figure out where ssh is listening.. But if you change your threat model and talk about bots? Logs are much cleaner and moving ports gets rid of a lot of traffic. Obviously so does enabling keys only.
Also does anyone still port knock these days?
Also does anyone still port knock these days?
Enter Masscan, probably a net negative for the internet, so use with care.
I didn't see anything about port knocking there, it rather looks like it has the opposite focus - a quote from that page is "features that support widespread scanning of many machines are supported, while in-depth scanning of single machines aren't."
Literally the only time I got somewhat hacked was when I left the default port of the service. Obscurity is reasonable, combined with other things like the ones mentioned here make you pretty much invulnerable to casuals. Somebody needs to target you to get anything.
Interesting. Do you know how it got compromised?
I published it to the internet and the next day, I couldn't ssh into the server anymore with my user account and something was off.
Tried root + password, also failed.
Immediately facepalmed because the password was the generic 8 characters and there was no fail2ban to stop guessing.
I canβt even figure out how to expose my services to the internet, honestly itβs probably for the best Wireguard gets the job done in the end.
I'm interested, how do you expose your services (on your PC I assume) to the internet through wireguard? Is it theough some VPN?
Wireguard IS a VPN. He has somehow through his challenges of exposing services to the internet, exposed wireguard from his home to the internet for him to connect to. Then he can connect to his internal services from there.
It's honestly the best option and how I operate as well. I only have a handful of items exposed and even those flow through a DMZ proxy before hitting their destination servers.
Oh, I thought it was a protocol for virtual networks, that merely VPNs used. The more you know!
Edit: spelled out VPN π
VPNβs are neat, besides the fact theyβre capable of masking your IP & DNS theyβre also capable of providing resources to devices outside a network.
A good example is the server at my work is only accessible on my works network, to access the server remotely without exposing it directly to the internet would be to use a VPN tunnel.
I'm having the opposite problem right now. Tightend a VM down so hard that now I can't get into it.
On a new linux install or image I will always:
- Make new users(s)
- Setup new user to sudo
- Change ssh port
- Change new user to authenticate ssh via key+password
- Disable root ssh login
- Setup new user to sudo
I hope it is not a passwordless sudo, it is basically the same as root.
That's more or less the advice I've gotten as well. I've also read good things about fail2ban which tries to ban sources of repeated authentication failures to prevent brute force password attempts. I've used it, but the only person who has managed to get banned is myself! I did get back in after the delay, but I'm happy to know it works.
How are people's servers getting compromised? I'm no security expert (I've never worked in tech at all) and have a public VPS, never been compromised. Mainly just use SSH keys not passwords, I don't do anything too crazy. Like if you have open SSH on port 22 with root login enabled and your root password is password123 then maybe but I'm surprised I've never been pwned if it's so easy to get got...
glad my root pass is toor and not something as obvious as password123
toor, like Tor, the leet hacker software. So it must be super secure.
The one db I saw compromised at a previous employer was an AWS RDS with public Internet access open and default admin username/password. Luckily it was just full of test data, so when we noticed its contents had been replaced with a ransom message we just deleted the instance.
That's incredible, I've got the same combination on my luggage.
By allowing password login and using weak passwords or by reusing passwords that have been involved in a data breach somewhere.
That makes sense. It feels a bit mad that the difference between getting pwned super easy vs not is something simple like that. But also reassuring to know, cause I was wondering how I heard about so many hobbyist home labs etc getting compromised when it'd be pretty hard to obtain a reasonably secured private key (ie not uploaded onto the cloud or anything, not stored on an unencrypted drive that other people can easily access, etc). But if it's just password logins that makes more sense.
And this is why every time a developer asks me for shell access to any of the deployment servers, I flat out deny the request.
Good on you for learning from your mistakes, but a perfect example for why I only let sysadmins into the systems.