[Advice needed] How to back up data across two SAMBA shares - rsync and crontab?

Support
#1

I’d like to periodically copy files from one Samba share (on USB disc 1) to another share (on USB disc 2). I’m not sure about the best way of setting this up, and whether installing additional software on the FreedomBox might interfere with its internal processes. I need some advice, please.

Can I just install rsync and set up a crontab job to do what I want? The usecase is copying media files weekly from one share to another (disc 1 to disc 2) to protect against hardware failure on disc 1. These are just plain portable USB drives, not RAID or anything fancy.

I’m running FreedomBox 20.12.1 on a Raspberry Pi 3 Model B, running Debian Buster.

rsync + crontab? Will this work or is there a better way?

Thanks!

1 Like
#2

You could give syncthing a try. (After attaching your backup disc 2 to another device.)

#3

I wondered about Syncthing too, but it assumes a second device. In my case, there’s only one device (the FreedomBox) but with two storage drives. I searched the Syncthing forums for “local sync” and it only appears as a feature request in 2018. A local sync option would be an ideal solution for me though.

Thanks for your suggestion.

#4

If it’s just to protect against a failing drive, an option may be to add your disc 2 as a second drive for the BTRFS filesystem of disc 1, and “balance” it as raid1 for data and metadata. https://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices#Conversion

#5

I will investigate that as a possibility. Thanks, @NickA . I have no experience with btrfs but it’s a good opportunity to learn. I’d hoped to only copy a subset of files to disc 2 (suits my needs better than full disc mirroring) but in the absence of a workable alternative, I may have to embrace RAID with more enthusiasm than I have to date.

#6

BTRFS filesystems work per partitions, so you could mount a separate new filesystem that only spans across two smaller partitions (residing on external or internal discs) to hold just the subdirectory with the subset of files that you want to reside on two discs.

After configuring btrfs once, I guess this kernel based solution could well be easier to use and be more reliable, day to day, than running and waiting for a separate application to watch and sync the files.

#7

Wow. That’s blown my mind a bit as a creative possibility. Agree that a kernel-based solution will likely be more reliable and easier to use than a separate application. I’ll do some more research and maybe dive in! :slight_smile:

#8

Just keep in mind that a raid provides instant replication, but not a backup.

Maybe the freedombox backup app will be able to keep backup versions of the files in your samba share (from the btrfs raid filesystem) somewhere else.

#9

You should also checkout rsnapshot + cron. It offers some nice features that you dont get with rsync alone. I’ve just implemented it myself, so can let you know how its working in a few days.

Syncing from 1 external samba share, to another external (plain. not a samba share). Also mostly for protection against HDD failure.

It was also relatively straightforward to set up.

#10

I think rsnapshot has the same problem with remote destinations as rsync, it can only work out its true efficiency when using the rsync protocol (rsync process on both sides), instead of simply accessing files on a remote filesystem share.

An alternative and more recent development seems to be borg backup. Freedombox uses it for its backup app, and there even exists a desktop client for it (https://vorta.borgbase.com/).

(Concerning my suggestion above, to use btrfs duplication (misnamed “raid 1”), that also only works as a local solution: Btrfs snapshots would allow to retain older versions, and btrfs send/receive would allow maintaining remote backup copies.)

1 Like
#11

Fair enough, although I don’t think the original question had remote shares. Just 2 externals.

I’d consider Borg as an option, but need to do more homework regarding btrfs and windows accessibility. Most devices on my network unfortunately run windows :neutral_face:

#12

Update: it seems to work quite well. Once I got cron properly configued

#13

Don’t you just use the samba shares to access the files on the server’s btrfs filesystem from windows clients? Or do you want external btrfs harddrives to be attachable to windows machines (win btrfs driver)?

#14

I wasn’t sure that was possible, so I just formatted them as NTFS. I’m a bit of a noob when it comes to filesystems :sweat_smile:

#15

Yes, and having a plain file backup has it’s own merrits. With rsync/rsnapshot, using the rsync protocol can speed it up considerably, though. :slight_smile: (Btrfs send can work over SSH, and even faster.)

1 Like