Did you talk about defaults for the new sharing feature already?
Looking at the current GUI mock-up, it seems it could still be much easier for all of the intended audience to have shares and groups readily configurable, allowing all freedombox users to easily collaborate on files by default.
So it may be good to have some sane safe defaults, that allow the users to collaborate out of the box, without requiring to set up all the above and custom groups.
What I have in mind is a default something like this:
Provide user authenticated shares over samba, when this can be considered save (maybe only for local nets?)
Make the /home directory browsable
Let /home also contain the user data of the defined groups, as for the default /home/group/users, /home/group/admins etc.
Let the users and group directories contain set-group-id sub-directories that implement inherent collaboration features (thanks to the user’s primary UPG and a 0002 umask).
( Example along the lines, but slightly improved from the Debian Wiki: )
This would allow the collaboration features to be discoverable and usable simply by browsing the filesystem, and without needing any setup by the user beyond opening the Freedombox network share. (And the network itself is usually also browseable with most operating systems.)
Many single board computers have very limited storage on the root device (SD card) and an have attached disks that are much larger and faster (USB disks). The proposal should account for this.
Don’t think the storage location needs to be related to the filesystem directory layout.
Where the SDCard is really to be kept as (readwrite) system drive, which isn’t a good advise due to its limited reliability I think, it’s enough to move /home and mount or symlink it from the external drive.
A preferable alternative may be an option to move the entire root filesystem, but keep a backup copy on SDCard in case the board can’t boot from the external storage (not possible/detached/broken).
One technical advantage I see over the current Freedombox/share/{group,home,public} implementation is that it only requires a single default “share” configuration (one path) to serve and access all the user data in /home. The users get access to their individual data and all shared data, according to the filesystem permissions of the default directories, and any custom group-permission directories created below /home.
It might even work to share (samba), export (NFS), webserve (HTTPS/webdav) out a common / folder, by default, that can be directly visible without any authentication. It could list the /home directory, and any others, and ask for credentials when the user tries to enter the directory.
The options in the sharing UI may then only make other directories of the filesystem available and visible (over selectable protocols maybe) on the default main share/export/webserverer/etc.
By using default filesystem (sgid) group directories the entire second page of the configuration UI shown above woudn’t be necessary, anymore.
The shares page would merely have provide a listing of the /home directory.
Also, when creating shares is based on filesystem paths, the mounting of additional filesystems would be a separate thing, and could be handled on a different UI page, while removable disks may even be auto-mounted and also be readily available for sharing (see concept document).
Just for convenient customization, the share-UI could allow to list (possibly even browse) the shared subdirectories, and allow to adjust the (default) permissions. It could also support to manage further ACL based directory permissions, if that’s desired or needed at all.
Here is a comparison of both storage schemes. There seem to be advantages for both.
Please add further points if you know some.
Concerning the proposed simplified UI mock-up, I think it could support both storage schemes.
The /home share can always be created, by default.
Creating a default “portable share” could be an option when mounting a removable storage without support for unix permissions that does not have a share folder yet. (Creating empty directories for all possible groups.)
The sub-directory listing could work the same on both type of shares, but only allowing the adjustments that are possible.
For portable storage, the properties would only allow to enable/disable the predefined home-share / open / freedombox-users subdirectories.
ad-hoc name-based sub-dir storage
fs-based group-dir storage
sharing space config corresponds to user/group setup
-
✓
works without separate configuration
-
✓
full permissions support
No (only home,users,open)
✓
one-to-one user collaboration out of the box
-
✓
works on storage w/o unix permissions (FAT)
✓
-
everything writable, if mounted locally
✓
- (for root yes)
allows consistent storage permissions, if mounted locally (e.g. backup / replica / server down / offline usage)
-
✓
features are portable between servers
✓
Yes (may require a tool to adjust uids/gids on server or disk, or to make a copy based on user/group-name based mapping
trivial to support in share UI
-
✓ (should work even without directory listing/adjusting support )
Click here for a table template to add further points.
I’ll try to add some concluded overview points of mine:
A default /home share with user-authentication can work well for a very broad, even quite sophisticated range of use-cases, without requiring any configuration or maintenance by the user.
The basics of UPGs and auto-creating personal user’s sharing- and group-subdirectories (/etc/skel & addgroup?) could all be made to work in Debian generally, even without any visible sharing UI (basic configuration steps here).
Access by applications that don’t run as a user process would continue be realized through ACLs (not directly visible).
A sharing UI that simply lists of all sgid-directories and their permissions in a share (and in a group/ sub-directory), can readily visualize the specific access possibilities for the user. The listing could allow to ease (filesystem) permission customization, if this is ever needed at all with good directory skeletons?
The “final touch” to be fool-proof comes from "inotifywatch"es on the group dirs, to automatically fix permissions of falsely (client-forced) permissions of SFTP uploads and moved files.
Access permissions for shares that are stored on non-unix filesystems would (still) require considerable abstract configuration on the sharing protocol level. The complicated configuration could be reduced, though, by
keeping the current admins/homes/public/read-only selection.
or, using the same general user/group/others scheme (but using filesystem mount options to restrict local and remote access to user/group/other).
In both cases, storing individual, user-specific data would still need the “homes” share setup, that maps the names of directories to user accounts and access permissions.
Nicely complementing initiatives:
A separate <mountpoint>/FreedomBox/ subdirectory on external/portable storage that can contain app specific data directories as well as all the user data under FreedomBox/home (on unix filesystems).
A central storage place for all the freedombox configuration under /home/freedombox/<absolute-path-in-rootfs>. It allows to nicely move or copy all configs to external/portable storage together with the user-data (e.g. <mountpoint>/FreedomBox/home/freedombox/etc/plinth/plinth.config).
Thank you very much @vexch, I think only with the help of your last question on the bug tracker it became much clearer to me why it always was rather disappointing to try out existing NAS distributions.
I hope my answer can also make it much easier to others here to make sense of and understand the motivation of this thread than my prior technical details:
Do you know any NAS software that uses the /home folder as a share or uses an UPG based private/group folder structure?
Unfortunately, I don’t know any readily configured NAS software, yet, that provides a user /home share that allows for simple group collaboration out of the box. FBX might be the first.
But I’ve seen such configurations, e.g. at schools (NFS or AFS? with corresponding SMB access), that is why it feels so disappointing to see how most NAS in the wild only seem to be used with a global public share and maybe one or two others without facilitating general user and group collaboration, because it’s too much of an administration hassle to set up groups and all needed combinations of shares and permissions.
[There shouldn’t be any unsolvable problem when sharing the server’s /home.] As long as the NAS drive isn’t actually mounted as /home on the clients, I think the server’s /home behaves just like a separate directrory tree on the client (while, where desired, it also allows the network-mounted-home-dir use-case).
BTW to add perspective: Maybe the idea of samba’s [homes] feature (https://www.howtogeek.com/howto/ubuntu/share-ubuntu-home-directories-using-samba/) could be described as “admin helper with user isolation” (the sharing space still needs to be administered separately), whereas a NAS /home fileserver that can be used as separate network sharing filesystem or as central /home server, may be the “complete admin simplification (made obsolete) that includes simplified user collaboration (resurrected debian UPG scheme)”. And the simplification would also cover the client side, as there is only one share/url/mount/network drive etc. that needs to be accessed or set up on clients.
So I hope the freedombox developers can appreciate the benefits of combining my successful experiments (to set up debian compatible user-private-groups on a freedombox) with a good default /home share configuration: Freedombox to become the first really-complete NAS solution, that supports direct individual user- and group-collaboration that starts on the universal filesystem level without any admin or user hassles (out-of-the-box).
I think I can also answer this better now.
Users will probably appreciate to be able to choose between:
Placing and using a separate sharing-storage directory on an external disk (i.e. create additional subdirecories as with the current solution).
Sharing the entire content of an external disk (i.e bind-mounting the external disk to place(s) below /home that corresponds to the desired user/group sharing permissions).
EDITED:
It would make sense to use the same bind-mount UI also for the sub-directories created with option one, to make the storage location available at place(s) below /home, i.e. to provide and manage the user access permissions.
Security Note:
Apps would continue to need and use the (invisible) ACLs to access their specific subdirectories in the user and group dirs (apps running under their own UID). For the users however all access permissions simply map directly to locations in the /home directory tree (UPG scheme).
Compare the user point of view with using Nextcloud:
How to administrate the NAS?
Not needed, all users and groups created get their sharing folders automatically.
How to use the NAS?
Simply browse the network, when prompted enter your freedombox user credentials, and work with files in the user and group folders. (smb/dns-sd).
If plinth could provide an online (web-based) filemanager (something like the former ajaxexplorer), this Freedombox NAS could already be a very workable substitute for Nextcloud, especially on low-power SBCs. (And all other FBX apps like syncthing etc. would also work with their parts of the /home filesystem.)
And the UI to add custom storage devices (disks) to the NAS?
Instead of picking the disk drives and selecting only from the user-only, single-group, and pubic sharing mode, the UI could allow to pick the disks and select from the /home directory tree (assigning the disk to individual users and group dirs, implying their filesystem access permissions). And a toggle could switch between sharing the whole disk or just specially created directories.
Found out that Ubuntu had actually already re-enabled the umask 002 about 10 years ago. So it allows easy collaboration of an arbitrary number of system user groups, and one-to-one file collaboration and exchange between users, on its filesystems (and network shares): “Simply” by creating the desired (set-group-id) group directories.
Why should Freedombox put any more effort in allowing users (and requiring admins) to configure custom samba shares (based on a peculiar and limited “open/fbx-group/user” abstraction layer)? Limit the sunk amount of work? Or rescue created code, by re-using/purposing the samba-share-config feature to manage external disk (bind)mounts, instead of samba share stanzas?
When the default umask 002 is to be expected back in Debian, anyway, to support it’s user-private-group scheme sooner or later? (and currently only costs a one-liner config)
Because FreedomBox might benefit too much? If multi-user filesystem collaboration becomes trivial and would allow for a significant integration of the (mostly generic) apps in FreedomBox?
Because adding additional network filesystem protocols (or first just enable manual configuration) would become trivial as well? (think NFS/sshfs/syncthing/… exports based on /home directories)
There is now a script to try out how hassle-free multiple users can work with files stored in group directories. (On systems with user-private-groups like the stock Debian and umask 002.)
Hi NickA,
I think it’s a great idea! I 'm convinced that there is great appreciation for your proposed new feature.
I have a Olimex FreedomBox Pioneer home server, with a 1TB Sata Harddisk
I would like FreedomBox as central storage place to save all data from my Phones (Android), Tablets (Ipad) to it.
I think your solution could make this happen
Let me know if you need a test user to test your idea, I’m willing to help!
If you like, you’re welcome to try the script, for example in a virtual machine. Group directories should be possible to set up with all distros that create users with user-private-groups. It would be great if you could post how groupdirs work for you.
It’s just the basic solution (a requirement actually) to manage the permissions that allow system users to make selected files also accessible and editable to other users and groups, if they want to. Simply by choosing a corresponding filesystem directory with their applications.
The network filesystem access can be build on top of it, and is a way to make good use of it for local machines.
For mobile devices that are only connected remotely at times, syncing the group directories with the freedombox (“Syncthing” app) should be much better then using samba network shares. It could also be better not to sync all your private user files and directories with a freedombox, that is running public services, and your mobile (security precaution). Instead it could make sense to only sync selected directories, and possibly sync all files only with a private NAS (e.g. a 2nd freedombox) that is only accessible in the local network.
The basis may be that a freedombox user syncs /home/<freedombox-user>/mobile-sync with a document subfolder on its devices, or even the entire user-data on the devices.
And sharing files with other user’s devices could be enabled with symbolic links.
For example, by creating /home/<freedombox-user>/mobile-sync/family that points to /home/group/family/mobile-sync
If syncing the entire data-partition on the device, this should then result simply in a sub-directory family/ (with all files in there automatically distributed to all family devices).
Note, how all the heavy permission stuff can work based on the universal filesystem based solution. And that such syncing is not quite feasible based on the current individual purely samba based usershare configurations (without utilizing the user-private-groups +home +groupdir scheme).
