Is my system being attacked?

Support
1

Hi all,
I have been using Freedombox for years. I just do the minimum to get it set up and then just forget about it.

My system went odd this weekend. It was connected to the internet, I could ping sights and DNS would resolve but no packets were returned. We had no access to any websites.

I looked for logs, but they were missing for hours over night. Several services were failed, I don’t remember which other than fail2ban.

I just reinstalled and everything went back to normal.

Then the same thing happened the next day. So I reinstalled and made fancier passwords.

Then it happened again today. I was able to look at the logs and was getting a lot of ssh errors and then services were failing. Again Fail2ban was not working.

I tried to reinstall, but as the machine was doing the first updates, the logs started saying ssh connections closed and unable to ? dieter-something. So I turned off ssh.

The freedombox is running now and is quiet.

Should I do something?

Thanks
Scot

2

Have you tried moving the SSH port to a non-standard port? This usually spares me the numerous “login attempts” that are (hopefully) blocked by fail2ban - reducing the attack vector.

3

Did your 10 JUN update go smoothly?

Lot of people had issues with the update - seems like you might have some packages that are trying to update or only partially updated and failing on attempt to complete?

4

I’ll do that if I turn SSH back on. I just don’t think that I really need it at this point.

1 Like
5

I really don’t know, it updates at 2 am and it was down in the morning, so maybe that was it.

Odd though that I reinstalled and updated it and it ran fine. Then it stopped letting traffic out. May it was just an issue with the software.

6

The night time-timing would be suspect to be for the updates.

You could kick off a background task via script to ping 1.1.1.1 or 8.8.8.8 every minute and dump the output to log file, then review tonight. See if the timing roughly correlates.

something like

#! /bin/sh
export LANG=C TZ=UTC
( date; ping -c1 server.com ) >> /path/to/server.log

and set a cron job to execute every minute

7

To me, it sounds less like and attack and likely issue caused by an update especially due to the timing and repeat behavior.

8

System went down again.
Here is the logs

June 14, 2023
9:54 PM
plinth.service: Failed with result ‘exit-code’.
systemd
14
9:54 PM
pam_ssh_add: Failed adding some keys
cockpit-session
9:45 PM
plinth.service: Failed with result ‘exit-code’.
systemd
91
9:45 PM
fail2ban.service: Failed with result ‘exit-code’.
systemd
9:45 PM
Failed to start firewalld - dynamic firewall daemon.
systemd
9:45 PM
firewalld.service: Failed with result ‘exit-code’.
systemd
9:45 PM
plinth.service: Failed with result ‘exit-code’.
systemd
9:45 PM
Failed to start Entropy Daemon based on the HAVEGE algorithm.
systemd
9:45 PM
haveged.service: Failed with result ‘signal’.
systemd
9:45 PM
haveged.service: Start request repeated too quickly.
systemd
9:45 PM
fail2ban.service: Failed with result ‘exit-code’.
systemd
9:45 PM
haveged.service: Failed with result ‘signal’.
systemd
9:45 PM
haveged.service: Main process exited, code=killed, status=31/SYS
systemd
9:45 PM
haveged.service: Failed with result ‘signal’.
systemd
9:45 PM
haveged.service: Main process exited, code=killed, status=31/SYS
systemd
9:45 PM
haveged.service: Failed with result ‘signal’.
systemd
9:45 PM
haveged.service: Main process exited, code=killed, status=31/SYS
systemd
9:45 PM
haveged.service: Failed with result ‘signal’.
systemd
9:45 PM
haveged.service: Main process exited, code=killed, status=31/SYS
systemd
9:45 PM
haveged.service: Failed with result ‘signal’.
systemd
9:45 PM
haveged.service: Main process exited, code=killed, status=31/SYS
systemd
9:43 PM
pmlogger_daily-poll.timer: Failed with result ‘resources’.
systemd
9:43 PM
pmlogger_daily-poll.timer: Unit to trigger vanished.
systemd
9:38 PM
/lib/systemd/system/pmie_check.service:9: Unit configured to use KillMode=none. This is unsafe, as it disables systemd’s process lifecycle management for the service. Please update your service to use a safer KillMode=, such as ‘mixed’ or ‘control-group’. Support for KillMode=none is deprecated and will eventually be removed.
systemd
9:38 PM
/lib/systemd/system/pmie_daily.service:8: Unit configured to use KillMode=none. This is unsafe, as it disables systemd’s process lifecycle management for the service. Please update your service to use a safer KillMode=, such as ‘mixed’ or ‘control-group’. Support for KillMode=none is deprecated and will eventually be removed.
systemd
9:38 PM
/lib/systemd/system/pmlogger_check.service:8: Unit configured to use KillMode=none. This is unsafe, as it disables systemd’s process lifecycle management for the service. Please update your service to use a safer KillMode=, such as ‘mixed’ or ‘control-group’. Support for KillMode=none is deprecated and will eventually be removed.
systemd
9:38 PM
/lib/systemd/system/pmlogger_daily.service:8: Unit configured to use KillMode=none. This is unsafe, as it disables systemd’s process lifecycle management for the service. Please update your service to use a safer KillMode=, such as ‘mixed’ or ‘control-group’. Support for KillMode=none is deprecated and will eventually be removed.
systemd
9:38 PM
/lib/systemd/system/pmlogger_daily-poll.service:8: Unit configured to use KillMode=none. This is unsafe, as it disables systemd’s process lifecycle management for the service. Please update your service to use a safer KillMode=, such as ‘mixed’ or ‘control-group’. Support for KillMode=none is deprecated and will eventually be removed.
systemd
9:38 PM
[2edbe4] <group/member=“Debian-exim”> no available LDAP server found: Server is unavailable: Connection refused
nslcd
2
9:38 PM
[9554fe] <group/member=“openldap”> no available LDAP server found: Server is unavailable: Connection refused
nslcd
9:38 PM
[9554fe] <group/member=“openldap”> no available LDAP server found: Can’t contact LDAP server: Connection refused
nslcd
9:38 PM
[0ea5d1] <group/member=“openldap”> no available LDAP server found: Can’t contact LDAP server: Connection refused
nslcd
9:38 PM
[0ea5d1] <group/member=“openldap”> failed to bind to LDAP server ldapi:///: Can’t contact LDAP server: Connection refused
nslcd
9:37 PM
[9554fe] <group/member=“openldap”> no available LDAP server found, sleeping 1 seconds
nslcd
9:37 PM
[9554fe] <group/member=“openldap”> failed to bind to LDAP server ldapi:///: Can’t contact LDAP server: Connection refused
nslcd
9:37 PM
[0ea5d1] <group/member=“openldap”> no available LDAP server found, sleeping 1 seconds
nslcd
9:37 PM
[0ea5d1] <group/member=“openldap”> failed to bind to LDAP server ldapi:///: Can’t contact LDAP server: Connection refused
nslcd
9:37 PM
[9554fe] <group/member=“openldap”> no available LDAP server found, sleeping 1 seconds
nslcd
9:37 PM
[9554fe] <group/member=“openldap”> failed to bind to LDAP server ldapi:///: Can’t contact LDAP server: Connection refused
nslcd
9:37 PM
[0ea5d1] <group/member=“openldap”> no available LDAP server found, sleeping 1 seconds
nslcd
9:37 PM
[0ea5d1] <group/member=“openldap”> failed to bind to LDAP server ldapi:///: Can’t contact LDAP server: Connection refused
nslcd
9:37 PM
[0ea5d1] <group/member=“openldap”> no available LDAP server found: Can’t contact LDAP server: Connection refused
nslcd
9:37 PM
[0ea5d1] <group/member=“openldap”> failed to bind to LDAP server ldapi:///: Can’t contact LDAP server: Connection refused
nslcd
9:37 PM
[9554fe] <group/member=“openldap”> no available LDAP server found, sleeping 1 seconds
nslcd
9:37 PM
[9554fe] <group/member=“openldap”> failed to bind to LDAP server ldapi:///: Can’t contact LDAP server: Connection refused
nslcd
9:37 PM
[0ea5d1] <group/member=“openldap”> no available LDAP server found, sleeping 1 seconds
nslcd
9:37 PM
[0ea5d1] <group/member=“openldap”> failed to bind to LDAP server ldapi:///: Can’t contact LDAP server: Connection refused
nslcd
9:37 PM
[9554fe] <group/member=“openldap”> no available LDAP server found, sleeping 1 seconds
nslcd
9:37 PM
[9554fe] <group/member=“openldap”> failed to bind to LDAP server ldapi:///: Can’t contact LDAP server: Connection refused
nslcd
9:37 PM
[0ea5d1] <group/member=“openldap”> no available LDAP server found, sleeping 1 seconds
nslcd
9:37 PM
[0ea5d1] <group/member=“openldap”> failed to bind to LDAP server ldapi:///: Can’t contact LDAP server: Connection refused
nslcd
9:37 PM
[9554fe] <group/member=“openldap”> no available LDAP server found, sleeping 1 seconds
nslcd
9:37 PM
[9554fe] <group/member=“openldap”> failed to bind to LDAP server ldapi:///: Can’t contact LDAP server: Connection refused
nslcd
9:37 PM
[0ea5d1] <group/member=“openldap”> no available LDAP server found, sleeping 1 seconds
nslcd
9:37 PM
[0ea5d1] <group/member=“openldap”> failed to bind to LDAP server ldapi:///: Can’t contact LDAP server: Connection refused
nslcd
9:37 PM
[9554fe] <group/member=“openldap”> no available LDAP server found, sleeping 1 seconds
nslcd
9:37 PM
[9554fe] <group/member=“openldap”> ldap_search_ext() failed: Can’t contact LDAP server: Broken pipe
nslcd
9:37 PM
[0ea5d1] <group/member=“openldap”> no available LDAP server found, sleeping 1 seconds
nslcd
9:37 PM
[0ea5d1] <group/member=“openldap”> failed to bind to LDAP server ldapi:///: Can’t contact LDAP server: Connection refused
nslcd
9:37 PM
[0ea5d1] <group/member=“openldap”> no available LDAP server found, sleeping 1 seconds
nslcd
9:37 PM
[0ea5d1] <group/member=“openldap”> failed to bind to LDAP server ldapi:///: Can’t contact LDAP server: Connection refused
nslcd
9:37 PM
[0ea5d1] <group/member=“openldap”> no available LDAP server found, sleeping 1 seconds
nslcd
9:37 PM
[0ea5d1] <group/member=“openldap”> failed to bind to LDAP server ldapi:///: Can’t contact LDAP server: Connection refused
nslcd
9:37 PM
[0ea5d1] <group/member=“openldap”> no available LDAP server found, sleeping 1 seconds
nslcd
9:37 PM
[0ea5d1] <group/member=“openldap”> failed to bind to LDAP server ldapi:///: Can’t contact LDAP server: Connection refused
nslcd
9:37 PM
[0ea5d1] <group/member=“openldap”> no available LDAP server found, sleeping 1 seconds
nslcd
9:37 PM
[0ea5d1] <group/member=“openldap”> failed to bind to LDAP server ldapi:///: Can’t contact LDAP server: Connection refused
nslcd
9:37 PM
[0ea5d1] <group/member=“openldap”> no available LDAP server found, sleeping 1 seconds
nslcd
9:37 PM
[0ea5d1] <group/member=“openldap”> ldap_search_ext() failed: Can’t contact LDAP server: Broken pipe
nslcd
9:35 PM
/lib/systemd/system/pmie_check.service:9: Unit configured to use KillMode=none. This is unsafe, as it disables systemd’s process lifecycle management for the service. Please update your service to use a safer KillMode=, such as ‘mixed’ or ‘control-group’. Support for KillMode=none is deprecated and will eventually be removed.
systemd
9:35 PM
/lib/systemd/system/pmie_daily.service:8: Unit configured to use KillMode=none. This is unsafe, as it disables systemd’s process lifecycle management for the service. Please update your service to use a safer KillMode=, such as ‘mixed’ or ‘control-group’. Support for KillMode=none is deprecated and will eventually be removed.
systemd
9:35 PM
/lib/systemd/system/pmlogger_check.service:8: Unit configured to use KillMode=none. This is unsafe, as it disables systemd’s process lifecycle management for the service. Please update your service to use a safer KillMode=, such as ‘mixed’ or ‘control-group’. Support for KillMode=none is deprecated and will eventually be removed.
systemd
9:35 PM
/lib/systemd/system/pmlogger_daily.service:8: Unit configured to use KillMode=none. This is unsafe, as it disables systemd’s process lifecycle management for the service. Please update your service to use a safer KillMode=, such as ‘mixed’ or ‘control-group’. Support for KillMode=none is deprecated and will eventually be removed.
systemd
9:35 PM
/lib/systemd/system/pmlogger_daily-poll.service:8: Unit configured to use KillMode=none. This is unsafe, as it disables systemd’s process lifecycle management for the service. Please update your service to use a safer KillMode=, such as ‘mixed’ or ‘control-group’. Support for KillMode=none is deprecated and will eventually be removed.
systemd
3:46 PM
Ignoring domain ministrywithcommunity.local for DHCP host name M-1754
dnsmasq-dhcp
24
3:28 PM
pam_ssh_add: Failed adding some keys
cockpit-session
8:08 AM
Ignoring domain ministrywithcommunity.local for DHCP host name M-1754
dnsmasq-dhcp
27
7:48 AM
pam_ssh_add: Failed adding some keys
cockpit-session
12:10 AM
Ignoring domain ministrywithcommunity.local for DHCP host name M-1754

June 14, 2023
10:01 PM
Failed to start firewalld - dynamic firewall daemon.
systemd
10:01 PM
Failed to start Entropy Daemon based on the HAVEGE algorithm.
systemd
Reboot

I can’t get it up again. I don’t know what to do
Scot

9

So I have been watching the logs and see the same stuff over and over, here is a bit of this morning when internet was down

7:38 AM fatal: Timeout before authentication for 61.177.172.140 port 63810 sshd
7:33 AM fatal: Timeout before authentication for 139.59.7.115 port 48044 sshd
7:28 AM fatal: Timeout before authentication for 34.131.225.98 port 51210 sshd
7:28 AM fatal: Timeout before authentication for 180.101.88.247 port 25583 sshd
7:22 AM error: kex_exchange_identification: Connection closed by remote host sshd
7:21 AM fatal: Timeout before authentication for 218.92.0.24 port 50528 sshd
7:16 AM fatal: Timeout before authentication for 218.92.0.25 port 35049 sshd
7:15 AM fatal: Timeout before authentication for 189.57.151.124 port 60647 sshd
7:09 AM fatal: Timeout before authentication for 218.92.0.107 port 31643 sshd
7:08 AM fatal: Timeout before authentication for 54.37.228.73 port 37546 sshd
7:07 AM fatal: Timeout before authentication for 34.131.225.98 port 52548 sshd
7:07 AM fatal: Timeout before authentication for 74.208.125.27 port 36712 sshd
7:02 AM fatal: Timeout before authentication for 54.39.165.222 port 52852 sshd
7:02 AM fatal: Timeout before authentication for 91.103.252.38 port 38656 sshd
6:55 AM fatal: Timeout before authentication for 189.57.151.124 port 34402 sshd
6:53 AM fatal: Timeout before authentication for 20.232.30.249 port 57798 sshd
6:51 AM error: kex_exchange_identification: Connection closed by remote host sshd

It just repeats for hours.
https://www.abuseipdb.com/check/218.92.0.118

All seem to be from here ChinaNet Jiangsu Province Network

Services keep failing and I am having my system go down. I am not anything in their league.
I thought I shut down ssh, but it was on again this morning.
Any idea of what to do/

10

Move SSH to a different port. Something high and obscure like 8989.

Or, if you can, use two NICs to split your FB networking into two zones and only allow SSH from internal.

11

Isn’t it more efficient to put one or more ssh public keys and disable password usage for login?

12

If he’s getting drowned in ssh login requests to the point his device is going under, then my thought would be to drop all that traffic entirely. Turn off port 22 and drop all packets to it, and go to another port.

Moving ssh ports is a very common recommendation and it doesn’t change the current working process very much while defeating the easy, low-hanging fruit brute forcing that the machine appears to be suffering under.

A ssh key is nice but we get a lot of people here who are first time comers to managing their own systems. I try to keep the first line recommendations simple just in case. Port change is easy to do and doesn’t carry a risk of accidental lock out if the key is misplaced or lost (ie a computer disk failure).

But @RodsThenCones if you want to add a key and go key auth, by all means do so. Its terribly convenient at the least. Just backup your private key (the one without the .pub ending) somewhere safe and please, for the love of all that is holy, not a repo on github.

1 Like
13

I would just add to use a random passphrase, i.e. a random list of words generated by your computer (not online, keepassxc works well). I use 10 words, with time I have become really fast typing it. It is easier if it is in the language you are most comfortable with (I don’t use English, it took me a bit of time to get a good list to generate it from).

14

KeepassXC is good stuff.

15

Thanks for all the advice.

I am gonna just shut down ssh for now. I doubt that I will need to log in remote.
I realized that after all of my troubles with the update, that my system was already compromised. I have several login attempts with odd usernames and then what seems like frustration that it didn’t work. Then I get a lot of attempts to log in.
I have realized that I need to look at config files and listen to error logs.
I’ll get there, so thanks for the help.
Scot

16

Wipe and have a new go at it sounds like a good plan. Sometimes, I keep stabbing at things I should have just given up on an hour or two previous.

It was frustrating at times for me in the beginning as well. I was lucky in that, through work, I got some free servers and I could make VMs, back them up constantly, and break things repeatedly.

When I first started up a Freedombox, it was on a fresh Debian VM. I wrecked that thing so many times in a week or so, trying to get matrix federation working. Gave up on federation for the short term until I could learn more but the ability to snapshot my virtual machine before I did the big dumb was invaluable.

You can try it yourself either using a spare machine (if it has Intel vPro/AMD equivalent) running Proxmox Community edition or via Linode/Digital Ocean. The smallest droplet isn’t big but its cheap per month, even with a spare snapshot or two in storage, to tinker with. You get to be fearless when a 2min snapshot restore can undo whatever you did.

2 Likes