Chain HDDs / SSDs together to form one filesystem without LVM?
 

Hi,

I have an age-old problem where I'm doing backup and I don't have one hard drive that is large enough for the whole backup.
I want to be able to mount several ones at the same time, to form one large filesystem.
I can do that with LVM, but if one of the devices dies, the whole filesystem gets trashed. I need a system where files are written first on one hard drive, then on the next one and in the case if one hard drive is missing, I can still access the files on the existing one.

Is there such a thing and how is that called? Is that what they call JBOD? (after years I still can't figure out what that is supposed to be, it feels like JBOD means nothing at all, just that you have "a bunch of disks")

You can use MD RAID.

Never mind; I did not read the full requirement. I don't know if there is a type of filesystem that gives you that much control over disk block allocation AND continues running when half of its storage is broken.

Generally, filesystems tend to keep data that belongs to a file together physically. I remember this from the dawns of UNIX and McKusick's filesystem code, which used the concept of "cylinder groups" for that purpose. This behaviour enables you to recover a good deal of your files in the case of a head crash, but it's no guarantee how much you can recover. And a crashed filesystem should definitely not be used anymore.

I used MD RAID 5 and love it, but then I need 3 drives, not just 2. PLUS: it is not fully fault tolerant, if I lose more than one drive, it is game over.
MD RAID on Linux is super solid and reliable.

Now I don't want to use a typical file system to span 2 drives, because you are right, no filesystem will tolerate part of it suddenly missing. A better approach is to format both drives as EXT4 and then use something that spans them together as one filesystem, like mounting them in the same mount point and merging it, there might be something like that. I know that some filesystems allow you to mount a RW portion on a read only filesystem, in order to allow modifications, the live distributions do it like that.
I can't remember what is called.

That is a stackable f/s, not really what you are looking for. There have been various incantations of them - one being mergerfs which promises close to what you want. Haven't tried it.

As a passing comment, if you are running into such a brick wall, I'd suggest you need to redesign you backup regime for a tad more flexibilty.

You could really mount any thing to a filesystem to make it transparent to the user I suppose.

A similar thing to LVM is ZFS which has many features beyond LVM. Suppose BtrFS can do it maybe.

I'm using rsync for my backups and that is already very flexible, in that it is using a no-format storage format.
I have just had this problem for decades now and I can't believe that I have not found a solution for it, now is the time to fix this, solve this, once and for all.
Thanks for the tip with mergefs, will take a look at it, it might solve the problem.

Yes, mergefs will do the trick.

As they write on their github page, it supports this very useful feature:
Handling of writes to full drives (transparently move file to drive with capacity)

Now it is just a matter of how to make it work, how to configure it.