Suddenly unable to log in to any account

Hardware Owners Support
1
  • FreedomBox is plugged into a router at home
  • The month and year you bought your kit: Q4 2024
  • The version of FreedomBox you are running: Unknown, as it was set to auto-update

Hello all,

I am running into an issue that I am hoping someone can help/assist with.

I am currently running a Freedombox Pioneer Edition from Olimex. It is running a fairly recent version, as I’ve set it to auto update.

This morning, I found that my ejabberd server was unreachable, so I attempted to log in to the system to take a look and was unable to do so.

When I attempt to access the freedombox via the web interface, I receive the following error:

This site can’t be reached
refused to connect.

When I attempt to log in via SSH using PuTTY, I receive the following error:

Remote side unexpectedly closed network connection

If I attempt to access the system again via PuTTY, I immediately get the closed network connection error again.

If I wait for a couple of minutes, I am allowed to attempt another login. I am unable to log in as the admin or as the local user, as it is not accepting my passwords.

I’ve also attempted to hard restart the system and can see it is booting, but it gets to a “starting kernel” prompt, then stops sending a signal to the monitor; I am not presented a login prompt.

I’ve tried to access XMPP to see if the service is even running but am unable to do so.

I use a password safe to make sure I am not mistyping them. Additionally, I was able to log in just fine to all my accounts using my saved passwords with no issues as of two days ago.

I don’t have another ARM system and, from my understanding, I am unable to boot the Freedombox Pioneer Edition via USB, so I cannot CHROOT into the SD Card to try and reset the password (though I don’t believe this is the root issue).

I also do not have a backup, as I was under the impression that the system backed itself up via the SD Card (I’ll make sure to set up Syncthing if/when I get it up and running again).

I’ve been using it for a while and would really like to not have to reimage to start over, as I’ve friends and family using the ejabberd server and have a running chat history. Additionally, I also do not want to lose my Let’s Encrypt certs.

Can someone point me in the right direction on what could be an issue (log files, for example) and/or how to fix?

Thanks!

2

On your router, can you see the locally assigned IP addresses, and see the one of your freedombox? If so, you may try pinging it with that local IP address, and try accessing the web page or ssh via this address. (It already happended to me that I was reaching the wrong machine.)

If that does not work, my suggestion would be to put the microSD card into a card reader and check the file systems there.

3

Hi Avron! I’ve attempted to access the FreedomBox via its local IP number and receive the errors mentioned above. I am able to ping it successfully. I tried your other suggestion and accessed the microSD card from an Ubuntu system. I can see the file system with no issues. Perhaps I can access a log or some other file from here that could help point us in the right direction? Thanks!

4

By default the journal is volatile which means, only written to memory, in order to avoid frequently writing to the microSD card, which is not so good for making a microSD card last for a long time.

There is a setting in configuration in the web interface to change that to persistent, but if you can’t access that, I guess the only way to do this would be to power off the device, modify the related configuration file on the microSD card from your Ubuntu computer, put it back to your device and keep it on for a while, then power it off and examine the journal from your Ubunutu computer.

I understand that is controlled by /etc/systemd/journald.conf.d/50-freedombox.conf, when I set this to persistent on my freedombox, that file only contains:

[Journal]
Storage=persistent

Perhaps it is also necessary to manually create the /var/log/journal directory.

For accessing the journal of another computer, reading the manual page of journalctl, journalctl -D path_of_the_var_log_journal_directory_of_the_microSD_card on your Ubunutu computer should allow you to see what was written to the journal. I never tried this so I could have misunderstood.

Still, be aware that it may difficult to find something useful in the journal.

5

In my undertanding, getting a new Let’s encrypt certificate should not result in any inconvenience, but yes, the chat history would be lost with a new installation. That said, clients may have the chat history anyway.

6

@DonValor I’ve experienced two cases where login attempts were failing. Both of these cases were when accessing the Pioneer on the local network. In the first case adding the domain name and ip address of the Pioneer to my /etc/hosts file enabled successful login when using the domain name to connect in the browser (and maybe ssh, don’t remember). This would have something to do with proper DNS resolution.

The other case was explored and resolved here: [SOLVED] Initial ssh session freezes, then new connection attempt fail

I think I power cycled the Pioneer to regain console access within the web interface in order to work the second case. Once in the console I was able to monitor some logging with “sudo journalctl -f” and “sudo journalctl -f -u ssh”.

rickbol

7

Thank you for all the help/feedback, everyone!

I went ahead and added the domain name to the local hosts file, but ended up receiving the same errors.

I mounted the SD Card on my Ubuntu system and was able to successfully move the files in /var/lib/abl/ to ~/abl-backup. After this, I booted the FreedomBox and was able to ping it successfully. However, I was not able to navigate to the plinth page, as it would still show the “This site can’t be reached, refused to connect.” error.

When I try to SSH as admin, it does NOT abruptly close the network connection. It now shows me an “Access Denied” error.

I’m going to try and see if I can get a persistent journal going. It may not be much help, but it won’t hurt either (as I don’t plan on keeping it persistent once/if the issue is resolved) :smiley:

8

Took a look at the journal… nothing much… Oh, well…

9

Here’s some instructions (from Sunil) for turning up debug logging on ssh:

  1. Drop a configuration file into /etc/ssh/sshd_config.d/

  2. Have just one line “LogLevel DEBUG” in the file. See /etc/ssh/sshd_config for reference.

  3. echo “LogLevel DEBUG” > /etc/ssh/sshd_config.d/mydebug.conf

  4. systemctl restart sshd

BTW, LogLevel values of DEBUG2 and DEBUG3 produce greater output, if necessary.

10

I’ll get to work on that! Thanks, rickbol!