Hi,

I'm wanting to set up a "selective" overlayfs/unionfs that, when a file is written, it will normally be written to the "top" filesystem but if the filename matches a particular pattern then it is written directly to the "bottom" filesystem. Is this possible and, if so, how?

The reason why I ask is because I want to have a common /home partition across multiple systems, but I want to have all the subdirectories of the home directories that start with a dot to be kept in a separate directory that is specific to each system. This is because the directories that start with a dot are usually used for configuration files so should not be shared between multiple systems.

Thanks,

onlyonemac

Not sure what you are doing. A directory that starts with a dot is for the most part considered a hidden by convention but can be used for configuration by convention also. There is nothing that I know of that prevents it from being used for any reason.

So we get back to the issue. You want to mount a filesystem and when in use you want it to selectively write to a given subdirectory based on some metric? I'll have to ask what is causing this write operation?

No. Let me explain it like this:

A normal OverlayFS works as follows:
- when we do a directory listing, we get the files from both the top and bottom directories, with those in the top directory taking precedence
- when we do a read, we get the file in the top directory, if present, otherwise the file in the bottom directory
- when we do a write, it writes to the top directory, even if the file is already present in the bottom directory

What I want is as follows:
- when we do a directory listing, we get the files from both the top and bottom directories, with those in the top directory taking precedence (same as with normal OverlayFS)
- when we do a read, we get the file in the top directory, if present, otherwise the file in the bottom directory (same as with normal OverlayFS)
- when we do a write, it writes to the top directory, even if the file is already present in the bottom directory, unless the file path matches a particular pattern, in which case it always writes to the bottom directory (different to normal OverlayFS)

In my case, the use thereof is because I'm wanting to share a home partition between multiple Linux systems but not have the configuration files in the user's home directory shared between those systems (as those may be incompatible between different Linux systems). The easiest way to distinguish between configuration files and normal files is by looking at if the file is a hidden file or directory in the root of the user's home directory. Thus I want to have an OverlayFS mount as I have described mounted on /home with something like /home-config on the system partition is the bottom directory, the home partition's root is the top directory, and any files which match the pattern "/*/.*" are written to the bottom directory (thus keeping the user's configuration files under /home-config/<username> on the system partition, but the user's other files on the home partition).

Maybe someone else following this question would have a better solution.


If I understand you correctly I get the feeling this sort of action will happen. (again sorry if I am not following you)
https://git.kernel.org/cgit/linux/ke.../overlayfs.txt


"Changes to underlying filesystems
---------------------------------

Offline changes, when the overlay is not mounted, are allowed to either
the upper or the lower trees.

Changes to the underlying filesystems while part of a mounted overlay
filesystem are not allowed. If the underlying filesystem is changed,
the behavior of the overlay is undefined, though it will not result in
a crash or deadlock."

As you can see there are a few options as to what is upper and what is lower. You can indeed change that on your various mounts for each distro.

I get the feeling that a set of file permissions or simply present different /home locations would be a way to go.

The other option that may work would be to use some script to edit info on each boot and shutdown. I've used that before to shape each different distro boot to what I need. You can rename or edit or copy.

I didn't quite follow your suggestion, but I think what you're trying to say is to have a separate home directory for each system and then mount the other systems' home directories in a different place? My current setup is something like that, but it is very awkward to use when the user has files spread across multiple home directories. What I didn't get is how file permissions or "changing the upper and lower directories" is going to help...

Also using a script is not an option as I want it to all work correctly even if one of the systems fails to start up or shut down correctly (i.e. to not have the system left in an inconsistent state or at any risk of having the configuration directories lost).

I think I understand the desired setup, but have no good ideas regarding implementation. My need is somewhat similar. My HDD contains four Linux installations, I have a home directory in each. I want to access my photos seamlessly from any of the installations.

The photos are all stored on a separate partition (/dev/sda6). /dev/sda6 is mounted at /mnt/Pictures in all four Linuxes. In each of my home directories, Pictures is a symlink to /mnt/Pictures. Whenever I download new photos from my camera, I save them to ~/Pictures - it doesn't matter which Linux I'm using, since the actual storage location is /mnt/Pictures.

I'm not seeing what your issue is. Just mount /dev/sda6 on /home/<username>/Pictures rather than on /mnt/Pictures, then you will have "seamless" access.

My issue is about "seamless access" of my entire home directory from multiple Linux systems without configuration clashes.

What I meant was to change or hide or move the configuration files back and forth on each boot. At boot time and or upon shutdown you'd have to either have a script to configure it as you need to to avoid issues with file. Otherwise you'd have to boot to a mini distro and have it modify the /home as you need to correctly set up the environment.

I have never tried this but maybe you could somehow set of a links to only those files that you want shared kind of a scheme to avoid conflicts. For each distro you have a minimal /home and send data to the main home and then create a link to each distro on each write operation. A lot of trouble no matter how you look at it.

So then all I have to do is wait for mom to accidentally unplug my computer when I'm not home and I'll have a big configuration file cleanup mess to sort out. No thanks, I don't want something that relies on a bunch of scripts being executed correctly.

lol it really frustrates me how you ask a question about something specific, then people want a use case so you give a use case and they try to fix the use case instead of answering the actual question.

You might try the people who write the code for overlayfs/unionfs and ask them to modify it to suit your needs.

I was going to look at modifying it myself, actually, but I was wondering if maybe there was already a way to do this, hence the original question.

Please tell me your skill level on something like that. I fear that even the people who wrote it may not be able to provide what you wish exactly.

I'd ask them for sure if it is possible even before I attempted it.

Why not run a script immediately after the overlay system is mounted that replaces the appropriate files in the upper filesystem with symlinks to the lower filesystem's files? If the appropriate files are all files that don't begin with a dot, a single script should work for all users

I think I would rather *not* have the entire visible contents of my home directory as symbolic links.

Yes sure it's possible. In whatever line they have which determines whether to get the file from the upper or the lower filesystem, you just add an additional test on the filename.