Again, I noticed that the last battery status message from my phone (via ejabberd) was at 2:33 UTC (it comes every few minutes normally), so this is approximately the time of the problem. Looking at the journal, this is at the nightly backup.
After hours of playing with journalctl (with sudo), I found several things:
-
on Oct 11 (and previous days as well), after the nightly backup, all services were started again and Freedombox was functional
-
on Oct 12, in the morning, I had lost all contact with the freedombox, so I removed the power and put it back on. The journal shows that after backup, bind and coturn were restarted, then the command to restart ejabberd is show, and then nothing until boot.
-
on Oct 13, in the morning, the web interface works ok but no service works. After some time, I asked to reboot, then all services work. The journal shows a failure in the backup, no trace of the backup post hooks in the journal, no service is started. In the web interface, services are not running, there is an entry for the backup as if it were successful, no notification or warning anywhere besides inside each service page and the diagnostics.
-
on Oct 14, same situation, the web interface works ok but no services works.
Here are extracts of journal that show all this:
Oct 11 (no problem that day)
Extract 1: the nightly backup and after
Oct 11 04:04:39 fbox /usr/bin/plinth[463]: Running backup_pre hooks Oct 11 04:04:39 fbox /usr/bin/plinth[463]: # backups create-archive --path '/media/8fa5a314-cb9b-11eb-96c2-020a0480e0fc::scheduled: daily: 2021-10-11:04:03' --comment '{{"type": "scheduled", "periods": ["daily"]}}' --paths /e> Oct 11 04:04:39 fbox sudo[2691]: plinth : PWD=/ ; USER=root ; COMMAND=/usr/share/plinth/actions/backups create-archive --path /media/8fa5a314-cb9b-11eb-96c2-020a0480e0fc::scheduled: daily: 2021-10-11:04:03 --comment {{"type> Oct 11 04:04:39 fbox sudo[2691]: plinth : (command continued) /etc/ejabberd/ejabberd.pem /etc/matrix-synapse/homeserver.yaml /etc/matrix-synapse/log.yaml /var/lib/matrix-synapse/homeserver.db /etc/matrix-synapse/homeserver.> Oct 11 04:04:39 fbox sudo[2691]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=110) Oct 11 04:05:38 fbox sudo[2691]: pam_unix(sudo:session): session closed for user root Oct 11 04:05:38 fbox /usr/bin/plinth[463]: Running backup_post hooks Oct 11 04:05:38 fbox /usr/bin/plinth[463]: # service start bind9 Oct 11 04:05:38 fbox slapd[946]: slap_global_control: unrecognized control: 1.3.6.1.4.1.4203.666.5.16 Oct 11 04:05:38 fbox sudo[2698]: plinth : PWD=/ ; USER=root ; COMMAND=/usr/share/plinth/actions/service start bind9 Oct 11 04:05:38 fbox sudo[2698]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=110) Oct 11 04:05:46 fbox systemd[1]: Started BIND Domain Name Server. Oct 11 04:05:46 fbox named[2702]: starting BIND 9.16.15-Debian (Stable Release) Oct 11 04:05:46 fbox named[2702]: running on Linux armv7l 5.10.0-9-armmp-lpae #1 SMP Debian 5.10.70-1 (2021-09-30) Oct 11 04:05:46 fbox named[2702]: built with '--build=arm-linux-gnueabihf' '--prefix=/usr' '--includedir=/usr/include' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disabl> Oct 11 04:05:46 fbox named[2702]: running as: named -f -u bind Oct 11 04:05:46 fbox named[2702]: compiled by GCC 10.2.1 20210110 Oct 11 04:05:46 fbox named[2702]: compiled with OpenSSL version: OpenSSL 1.1.1k 25 Mar 2021 Oct 11 04:05:46 fbox named[2702]: linked to OpenSSL version: OpenSSL 1.1.1k 25 Mar 2021 Oct 11 04:05:46 fbox named[2702]: compiled with libxml2 version: 2.9.10 Oct 11 04:05:46 fbox named[2702]: linked to libxml2 version: 20910 Oct 11 04:05:46 fbox named[2702]: compiled with json-c version: 0.15 Oct 11 04:05:46 fbox named[2702]: linked to json-c version: 0.15 Oct 11 04:05:46 fbox named[2702]: compiled with zlib version: 1.2.11 Oct 11 04:05:46 fbox named[2702]: linked to zlib version: 1.2.11 Oct 11 04:05:46 fbox named[2702]: ---------------------------------------------------- Oct 11 04:05:46 fbox named[2702]: BIND 9 is maintained by Internet Systems Consortium, Oct 11 04:05:46 fbox named[2702]: Inc. (ISC), a non-profit 501(c)(3) public-benefit Oct 11 04:05:46 fbox named[2702]: corporation. Support and training for BIND 9 are Oct 11 04:05:46 fbox named[2702]: available at https://www.isc.org/support Oct 11 04:05:46 fbox named[2702]: ---------------------------------------------------- Oct 11 04:05:46 fbox named[2702]: adjusted limit on open files from 524288 to 1048576 Oct 11 04:05:46 fbox named[2702]: found 2 CPUs, using 2 worker threads Oct 11 04:05:46 fbox named[2702]: using 2 UDP listeners per interface Oct 11 04:05:46 fbox named[2702]: using up to 21000 sockets Oct 11 04:05:46 fbox named[2702]: loading configuration from '/etc/bind/named.conf' Oct 11 04:05:46 fbox named[2702]: /etc/bind/named.conf.options:16: option 'dnssec-enable' is obsolete and should be removed Oct 11 04:05:46 fbox named[2702]: reading built-in trust anchors from file '/etc/bind/bind.keys' Oct 11 04:05:46 fbox named[2702]: looking for GeoIP2 databases in '/usr/share/GeoIP' Oct 11 04:05:46 fbox named[2702]: using default UDP/IPv4 port range: [32768, 60999] Oct 11 04:05:46 fbox named[2702]: using default UDP/IPv6 port range: [32768, 60999] Oct 11 04:05:46 fbox named[2702]: listening on IPv4 interface lo, 127.0.0.1#53 Oct 11 04:05:46 fbox named[2702]: listening on IPv4 interface eth0, 192.168.1.66#53 Oct 11 04:05:46 fbox named[2702]: listening on IPv4 interface tun0, 10.91.0.1#53 Oct 11 04:05:46 fbox named[2702]: IPv6 socket API is incomplete; explicitly binding to each IPv6 address separately Oct 11 04:05:46 fbox named[2702]: listening on IPv6 interface lo, ::1#53 Oct 11 04:05:46 fbox named[2702]: listening on IPv6 interface eth0, fe80::2272:e4ce:2d48:7bc2%2#53 Oct 11 04:05:46 fbox named[2702]: listening on IPv6 interface tun0, fe80::2283:af87:9fe2:53c9%3#53 Oct 11 04:05:46 fbox named[2702]: generating session key for dynamic DNS Oct 11 04:05:46 fbox named[2702]: sizing zone task pool based on 5 zones
Extract 2: after the nightly backup, and bind9 and coturn were started:
Oct 11 04:05:55 fbox /usr/bin/plinth[463]: # service start ejabberd Oct 11 04:05:55 fbox sudo[2721]: plinth : PWD=/ ; USER=root ; COMMAND=/usr/share/plinth/actions/service start ejabberd Oct 11 04:05:55 fbox sudo[2721]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=110) Oct 11 04:06:02 fbox systemd[1]: Started Erlang Port Mapper Daemon. Oct 11 04:06:02 fbox systemd[1]: Starting robust, scalable and extensible realtime platform (XMPP server + MQTT broker + SIP service)... Oct 11 04:06:02 fbox audit[2744]: AVC apparmor="ALLOWED" operation="open" profile="/usr/sbin/ejabberdctl" name="/sys/fs/cgroup/cgroup.controllers" pid=2744 comm="beam.smp" requested_mask="r" denied_mask="r" fsuid=116 ouid=0 Oct 11 04:06:02 fbox kernel: audit: type=1400 audit(1633925162.613:53): apparmor="ALLOWED" operation="open" profile="/usr/sbin/ejabberdctl" name="/sys/fs/cgroup/cgroup.controllers" pid=2744 comm="beam.smp" requested_mask="r" > Oct 11 04:06:02 fbox audit[2744]: AVC apparmor="ALLOWED" operation="open" profile="/usr/sbin/ejabberdctl" name="/sys/fs/cgroup/cgroup.controllers" pid=2744 comm="beam.smp" requested_mask="r" denied_mask="r" fsuid=116 ouid=0 Oct 11 04:06:02 fbox kernel: audit: type=1400 audit(1633925162.625:54): apparmor="ALLOWED" operation="open" profile="/usr/sbin/ejabberdctl" name="/sys/fs/cgroup/cgroup.controllers" pid=2744 comm="beam.smp" requested_mask="r" > Oct 11 04:06:02 fbox audit[2754]: AVC apparmor="ALLOWED" operation="open" profile="/usr/sbin/ejabberdctl" name="/proc/2754/fd/" pid=2754 comm="erl_child_setup" requested_mask="r" denied_mask="r" fsuid=116 ouid=116 Oct 11 04:06:02 fbox kernel: audit: type=1400 audit(1633925162.681:55): apparmor="ALLOWED" operation="open" profile="/usr/sbin/ejabberdctl" name="/proc/2754/fd/" pid=2754 comm="erl_child_setup" requested_mask="r" denied_mask=> Oct 11 04:06:04 fbox audit[2780]: AVC apparmor="ALLOWED" operation="open" profile="/usr/sbin/ejabberdctl" name="/sys/fs/cgroup/cgroup.controllers" pid=2780 comm="beam.smp" requested_mask="r" denied_mask="r" fsuid=116 ouid=0 Oct 11 04:06:04 fbox kernel: audit: type=1400 audit(1633925164.841:56): apparmor="ALLOWED" operation="open" profile="/usr/sbin/ejabberdctl" name="/sys/fs/cgroup/cgroup.controllers" pid=2780 comm="beam.smp" requested_mask="r" > Oct 11 04:06:04 fbox audit[2780]: AVC apparmor="ALLOWED" operation="open" profile="/usr/sbin/ejabberdctl" name="/sys/fs/cgroup/cgroup.controllers" pid=2780 comm="beam.smp" requested_mask="r" denied_mask="r" fsuid=116 ouid=0 Oct 11 04:06:04 fbox kernel: audit: type=1400 audit(1633925164.849:57): apparmor="ALLOWED" operation="open" profile="/usr/sbin/ejabberdctl" name="/sys/fs/cgroup/cgroup.controllers" pid=2780 comm="beam.smp" requested_mask="r" > Oct 11 04:06:04 fbox audit[2791]: AVC apparmor="ALLOWED" operation="open" profile="/usr/sbin/ejabberdctl" name="/proc/2791/fd/" pid=2791 comm="erl_child_setup" requested_mask="r" denied_mask="r" fsuid=116 ouid=116 Oct 11 04:06:04 fbox kernel: audit: type=1400 audit(1633925164.925:58): apparmor="ALLOWED" operation="open" profile="/usr/sbin/ejabberdctl" name="/proc/2791/fd/" pid=2791 comm="erl_child_setup" requested_mask="r" denied_mask=> Oct 11 04:06:10 fbox audit[2819]: AVC apparmor="ALLOWED" operation="open" profile="/usr/sbin/ejabberdctl" name="/sys/fs/cgroup/cgroup.controllers" pid=2819 comm="beam.smp" requested_mask="r" denied_mask="r" fsuid=116 ouid=0 Oct 11 04:06:10 fbox kernel: audit: type=1400 audit(1633925170.517:59): apparmor="ALLOWED" operation="open" profile="/usr/sbin/ejabberdctl" name="/sys/fs/cgroup/cgroup.controllers" pid=2819 comm="beam.smp" requested_mask="r" > Oct 11 04:06:10 fbox audit[2819]: AVC apparmor="ALLOWED" operation="open" profile="/usr/sbin/ejabberdctl" name="/sys/fs/cgroup/cgroup.controllers" pid=2819 comm="beam.smp" requested_mask="r" denied_mask="r" fsuid=116 ouid=0 Oct 11 04:06:10 fbox kernel: audit: type=1400 audit(1633925170.533:60): apparmor="ALLOWED" operation="open" profile="/usr/sbin/ejabberdctl" name="/sys/fs/cgroup/cgroup.controllers" pid=2819 comm="beam.smp" requested_mask="r" > Oct 11 04:06:10 fbox audit[2830]: AVC apparmor="ALLOWED" operation="open" profile="/usr/sbin/ejabberdctl" name="/proc/2830/fd/" pid=2830 comm="erl_child_setup" requested_mask="r" denied_mask="r" fsuid=116 ouid=116 Oct 11 04:06:10 fbox kernel: audit: type=1400 audit(1633925170.581:61): apparmor="ALLOWED" operation="open" profile="/usr/sbin/ejabberdctl" name="/proc/2830/fd/" pid=2830 comm="erl_child_setup" requested_mask="r" denied_mask=> Oct 11 04:06:16 fbox audit[2856]: AVC apparmor="ALLOWED" operation="open" profile="/usr/sbin/ejabberdctl" name="/sys/fs/cgroup/cgroup.controllers" pid=2856 comm="beam.smp" requested_mask="r" denied_mask="r" fsuid=116 ouid=0 Oct 11 04:06:16 fbox kernel: audit: type=1400 audit(1633925176.513:62): apparmor="ALLOWED" operation="open" profile="/usr/sbin/ejabberdctl" name="/sys/fs/cgroup/cgroup.controllers" pid=2856 comm="beam.smp" requested_mask="r" > Oct 11 04:06:16 fbox audit[2856]: AVC apparmor="ALLOWED" operation="open" profile="/usr/sbin/ejabberdctl" name="/sys/fs/cgroup/cgroup.controllers" pid=2856 comm="beam.smp" requested_mask="r" denied_mask="r" fsuid=116 ouid=0 Oct 11 04:06:16 fbox kernel: audit: type=1400 audit(1633925176.525:63): apparmor="ALLOWED" operation="open" profile="/usr/sbin/ejabberdctl" name="/sys/fs/cgroup/cgroup.controllers" pid=2856 comm="beam.smp" requested_mask="r" > Oct 11 04:06:16 fbox audit[2867]: AVC apparmor="ALLOWED" operation="open" profile="/usr/sbin/ejabberdctl" name="/proc/2867/fd/" pid=2867 comm="erl_child_setup" requested_mask="r" denied_mask="r" fsuid=116 ouid=116 Oct 11 04:06:16 fbox kernel: audit: type=1400 audit(1633925176.585:64): apparmor="ALLOWED" operation="open" profile="/usr/sbin/ejabberdctl" name="/proc/2867/fd/" pid=2867 comm="erl_child_setup" requested_mask="r" denied_mask=> Oct 11 04:06:20 fbox audit[2744]: AVC apparmor="ALLOWED" operation="open" profile="/usr/sbin/ejabberdctl" name="/etc/ssl/openssl.cnf" pid=2744 comm="1_scheduler" requested_mask="r" denied_mask="r" fsuid=116 ouid=0 Oct 11 04:06:20 fbox kernel: audit: type=1400 audit(1633925180.649:65): apparmor="ALLOWED" operation="open" profile="/usr/sbin/ejabberdctl" name="/etc/ssl/openssl.cnf" pid=2744 comm="1_scheduler" requested_mask="r" denied_mas> Oct 11 04:06:22 fbox audit[2894]: AVC apparmor="ALLOWED" operation="open" profile="/usr/sbin/ejabberdctl" name="/sys/fs/cgroup/cgroup.controllers" pid=2894 comm="beam.smp" requested_mask="r" denied_mask="r" fsuid=116 ouid=0 Oct 11 04:06:22 fbox kernel: audit: type=1400 audit(1633925182.229:66): apparmor="ALLOWED" operation="open" profile="/usr/sbin/ejabberdctl" name="/sys/fs/cgroup/cgroup.controllers" pid=2894 comm="beam.smp" requested_mask="r" > Oct 11 04:06:22 fbox audit[2894]: AVC apparmor="ALLOWED" operation="open" profile="/usr/sbin/ejabberdctl" name="/sys/fs/cgroup/cgroup.controllers" pid=2894 comm="beam.smp" requested_mask="r" denied_mask="r" fsuid=116 ouid=0 Oct 11 04:06:22 fbox kernel: audit: type=1400 audit(1633925182.241:67): apparmor="ALLOWED" operation="open" profile="/usr/sbin/ejabberdctl" name="/sys/fs/cgroup/cgroup.controllers" pid=2894 comm="beam.smp" requested_mask="r" > Oct 11 04:06:22 fbox audit[2905]: AVC apparmor="ALLOWED" operation="open" profile="/usr/sbin/ejabberdctl" name="/proc/2905/fd/" pid=2905 comm="erl_child_setup" requested_mask="r" denied_mask="r" fsuid=116 ouid=116 Oct 11 04:06:22 fbox kernel: audit: type=1400 audit(1633925182.309:68): apparmor="ALLOWED" operation="open" profile="/usr/sbin/ejabberdctl" name="/proc/2905/fd/" pid=2905 comm="erl_child_setup" requested_mask="r" denied_mask=> Oct 11 04:06:25 fbox systemd[1]: Started robust, scalable and extensible realtime platform (XMPP server + MQTT broker + SIP service). Oct 11 04:06:26 fbox sudo[2721]: pam_unix(sudo:session): session closed for user root Oct 11 04:06:26 fbox /usr/bin/plinth[463]: # service start mumble-server
and then other services started as well.
Oct 12: after the backup, restart of bind and restart of coturn
Oct 12 02:05:56 fbox /usr/bin/plinth[463]: # service start ejabberd Oct 12 02:05:56 fbox sudo[8359]: plinth : PWD=/ ; USER=root ; COMMAND=/usr/share/plinth/actions/service start ejabberd Oct 12 02:05:56 fbox sudo[8359]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=110) -- Boot 463ac2e38da5462abe6ad92effaef42a -- Oct 12 07:11:05 fbox kernel: Booting Linux on physical CPU 0x0 Oct 12 07:11:05 fbox kernel: Linux version 5.10.0-9-armmp-lpae (debian-kernel@lists.debian.org) (gcc-10 (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2) #1 SMP Debian 5.10.70-1 (2021-09-30)
On Oct 13: no service is running
Oct 13 02:12:23 fbox /usr/bin/plinth[500]: Running backup_pre hooks Oct 13 02:12:23 fbox /usr/bin/plinth[500]: # backups create-archive --path '/media/8fa5a314-cb9b-11eb-96c2-020a0480e0fc::scheduled: daily: 2021-10-13:02:11' --comment '{{"type": "scheduled", "periods": ["daily"]}}' --paths /e> Oct 13 02:12:23 fbox sudo[7230]: plinth : PWD=/ ; USER=root ; COMMAND=/usr/share/plinth/actions/backups create-archive --path /media/8fa5a314-cb9b-11eb-96c2-020a0480e0fc::scheduled: daily: 2021-10-13:02:11 --comment {{"type> Oct 13 02:12:23 fbox sudo[7230]: plinth : (command continued) /etc/ejabberd/ejabberd.pem /etc/matrix-synapse/homeserver.yaml /etc/matrix-synapse/log.yaml /var/lib/matrix-synapse/homeserver.db /etc/matrix-synapse/homeserver.> Oct 13 02:12:23 fbox sudo[7230]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=110) Oct 13 02:13:28 fbox sudo[7230]: pam_unix(sudo:session): session closed for user root Oct 13 02:13:28 fbox /usr/bin/plinth[500]: Error executing command - ['sudo', '-n', '/usr/share/plinth/actions/backups', 'create-archive', '--path', '/media/8fa5a314-cb9b-11eb-96c2-020a0480e0fc::scheduled: daily: 2021-10-13:0> "archive": { "command_line": [ "/usr/bin/borg", "create", "--json", "--comment", "{{\"type\": \"scheduled\", \"periods\": [\"daily\"]}}", "/media/8fa5a314-cb9b-11eb-96c2-020a0480e0fc::scheduled: daily: 2021-10-13:02:11", "/etc/ez-ipupdate/", "/etc/letsencrypt/", "/etc/coturn", "/var/lib/ejabberd/", "/etc/ejabberd/letsencrypt/", "/etc/matrix-synapse/conf.d/", "/var/lib/matrix-synapse/media/", "/var/lib/mumble-server", "/etc/openvpn/", "/var/lib/quassel/", "/etc/radicale/", "/var/lib/radicale/", "/etc/tor/", "/var/lib/tor/", "/var/lib/tor-instances/", "/etc/timezone", "/etc/security/access.d/50freedombox.conf", "/etc/snapper/configs/root", "/etc/default/snapper", "/etc/ssh/ssh_host_ecdsa_key", "/etc/ssh/ssh_host_ecdsa_key.pub", "/etc/ssh/ssh_host_ed25519_key", "/etc/ssh/ssh_host_ed25519_key.pub", "/etc/ssh/ssh_host_rsa_key", "/etc/ssh/ssh_host_rsa_key.pub", "/etc/apt/apt.conf.d/20auto-upgrades", "/etc/bind/named.conf.options", "/etc/ejabberd/ejabberd.yml", "/etc/ejabberd/ejabberd.pem", "/etc/matrix-synapse/homeserver.yaml", "/etc/matrix-synapse/log.yaml", "/var/lib/matrix-synapse/homeserver.db", "/etc/matrix-synapse/homeserver.signing.key", "/etc/matrix-synapse/homeserver.tls.crt", "/etc/matrix-synapse/homeserver.tls.key", "/var/lib/plinth/backups-manifests/media8fa5a314-cb9b-11eb-96c2-020a0480e0fcscheduled_daily_2021-10-130211.json" ], "duration": 33.10464, "end": "2021-10-13T02:13:06.000000", "id": "c67f9dcb85bcb480a70dbcc169bbcee81558aaabf5a072a4da56c40283e2423e", "limits": { "max_archive_size": 8.602159151483784e-05 }, "name": "scheduled: daily: 2021-10-13:02:11", "start": "2021-10-13T02:12:33.000000", "stats": { "compressed_size": 468886686, "deduplicated_size": 6103281, "nfiles": 5423, "original_size": 636725782 } }, "cache": { "path": "/root/.cache/borg/fb645b6f9e1cefa82bc08cc3ebe0de387307915777167b99f2e7a12af0d95c0d", "stats": { "total_chunks": 66436, "total_csize": 5584127171, "total_size": 7600532859, "total_unique_chunks": 8235, "unique_csize": 910532717, "unique_size": 1342580732 } }, "encryption": { "mode": "none" }, "repository": { "id": "fb645b6f9e1cefa82bc08cc3ebe0de387307915777167b99f2e7a12af0d95c0d", "last_modified": "2021-10-13T02:13:06.000000", "location": "/media/8fa5a314-cb9b-11eb-96c2-020a0480e0fc" } } , /var/lib/ejabberd/PREVIOUS.LOG: stat: [Errno 2] No such file or directory: '/var/lib/ejabberd/PREVIOUS.LOG' Traceback (most recent call last): File "/usr/share/plinth/actions/backups", line 310, in main() File "/usr/share/plinth/actions/backups", line 306, in main subcommand_method(arguments) File "/usr/share/plinth/actions/backups", line 144, in subcommand_create_archive run(command, arguments) File "/usr/share/plinth/actions/backups", line 296, in run return subprocess.run(cmd, check=check, env=env, **kwargs) File "/usr/lib/python3.9/subprocess.py", line 528, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['borg', 'create', '--json', '--comment', '{{"type": "scheduled", "periods": ["daily"]}}', '/media/8fa5a314-cb9b-11eb-96c2-020a0480e0fc::sched> Oct 13 02:13:28 fbox /usr/bin/plinth[500]: Error running scheduled backup: ('backups', '{\n "archive": {\n "command_line": [\n "/usr/bin/borg",\n "create",\n "--json",\n "> Traceback (most recent call last): File "/usr/lib/python3/dist-packages/plinth/modules/backups/__init__.py", line 119, in backup_by_schedule repository.schedule.run_schedule() File "/usr/lib/python3/dist-packages/plinth/modules/backups/schedule.py", line 205, in run_schedule self._run_backup(periods) File "/usr/lib/python3/dist-packages/plinth/modules/backups/schedule.py", line 297, in _run_backup repository.create_archive(name, app_ids, archive_comment=comment) File "/usr/lib/python3/dist-packages/plinth/modules/backups/repository.py", line 180, in create_archive api.backup_apps(_backup_handler, path=archive_path, app_ids=app_ids, File "/usr/lib/python3/dist-packages/plinth/modules/backups/api.py", line 130, in backup_apps _run_operation(backup_handler, packet, File "/usr/lib/python3/dist-packages/plinth/modules/backups/api.py", line 427, in _run_operation handler(packet, encryption_passphrase=encryption_passphrase) File "/usr/lib/python3/dist-packages/plinth/modules/backups/__init__.py", line 111, in _backup_handler actions.superuser_run('backups', arguments, input=input_data.encode()) File "/usr/lib/python3/dist-packages/plinth/actions.py", line 104, in superuser_run return _run(action, options, input, run_in_background, True, File "/usr/lib/python3/dist-packages/plinth/actions.py", line 200, in _run raise ActionError(action, output, error) plinth.errors.ActionError: ('backups', '{\n "archive": {\n "command_line": [\n "/usr/bin/borg",\n "create",\n "--json",\n "--com> Oct 13 02:17:01 fbox slapd[935]: slap_global_control: unrecognized control: 1.3.6.1.4.1.4203.666.5.16 Oct 13 02:17:01 fbox CRON[7240]: pam_env(cron:session): Unable to open env file: /etc/default/locale: No such file or directory Oct 13 02:17:01 fbox CRON[7240]: pam_unix(cron:session): session opened for user root(uid=0) by (uid=0) Oct 13 02:17:01 fbox CRON[7241]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) Oct 13 02:17:01 fbox CRON[7240]: pam_unix(cron:session): session closed for user root Oct 13 02:20:07 fbox systemd[1]: Started WordPress Scheduled Events Trigger (Cron). Oct 13 02:20:08 fbox php[7245]: Could not open input file: /usr/share/wordpress/wp-cron.php Oct 13 02:20:08 fbox systemd[1]: wordpress-freedombox.service: Main process exited, code=exited, status=1/FAILURE Oct 13 02:20:08 fbox systemd[1]: wordpress-freedombox.service: Failed with result 'exit-code'. Oct 13 02:30:07 fbox systemd[1]: Started WordPress Scheduled Events Trigger (Cron). Oct 13 02:30:08 fbox php[7256]: Could not open input file: /usr/share/wordpress/wp-cron.php Oct 13 02:30:08 fbox systemd[1]: wordpress-freedombox.service: Main process exited, code=exited, status=1/FAILURE
In summary: I have no clue what causes the problem after backup (on Oct 12, no more journal after the post backup hooks initiated restart of ejabberd) and/or of backup (on Oct 13 and 14, journal shows a backup failure and no post backup hooks, all services remain down).
Since the problem repeated twice the same, I expect it may continue (i.e. no service is working after backup), some help is welcome.