So, at the moment I have a 7TB LVM with 1 group and one logical volume.

In all honesty I don't back up this information. It is filled with data that I can "afford" to lose, but... would rather not.

How do LVMs fail? If I lose a 1.5TB drive that is part of the LVM does that mean at most I could lose 1.5TB of data? Or can files span more than one drive? if so, would it just be one file what would span two drives? or could there be many files that span multiple drives drives?

And so on...

Essentially. I'm just curious, in a general, in a high level sense about LVM safety. What are the risks that are involved?

Thanks!


Edit: what happens if I boot up the computer with a drive missing from the lvm? Is there a first primary drive?

LVM by provides no default redundancy. If you pvcreate several disks (PVs) then use vgcreate to put them in a VG then subdivide that with lvcreate into LVs you've not done any redunancy.

If one uses RAID devices as the PV then the redundancy is before the LVM layer. Losing a drive isn't really noticed by LVM in such a scenario (other than possible performance changes depending on RAID level.)

However, LVM does now offer mirroring at the LV level however. You can put multiple non-redundant disks in the the VG then just be sure that you mirror all LVs when you create them. (Type "man lvcreate" and "man lvchange" for more details of LV mirroring.)

I haven't used the LVM mirroring feature in Linux mainly because all the systems I support have hardware RAID controllers. However, LVM on HP-UX had an add on called MirrorUX and on all the systems I've loaded that UNIX OS on over the years I always installed that and setup the LVM mirroring which appears to be similar (if not the same) to the way Linux LVM mirroring works.

If you try and activate a volume group with a missing PV it'll complain and fail to activate. You can force it online, in which case you should be able to rescue any LVs that don't have any extents allocated on the missing disk. but any LVs that do will remain inaccessible.

If you've allocated all of that 7TB to a single LV and 1.5TB of that goes missing you'll have a 1.5TB hole in the structure of the filesystem on that LV. I'm fairly confident that in such a case you can say goodbye to the entire 7TB!

Ouch.

I think I might make a move away from lvm in that case. If I were keeping full backups it wouldn't be an issue. But this is an issue.

I don't plan on backing up all the data, but, it would be nice to minimize the loss.

Thanks

Yeah, this would be ideal.. but.. with my funds and such... just not happening. hehe.

Thanks

Yep. Probably a wise move. The problem you've got with your current setup is that you're effectively treating all your storage as one big disk, so you're in a lose one, lose all situation, in much the same way you'd be if you'd used raid-0 linear/striping.

I didn't quite understand. Moving away from LVM isn't going to save you anything if you're not doing redundancy of some sort. No matter how you do redundancy you'll end up needing to use up some disk space for it.

The cheapest way to do it would be to make yourself a RAID 5 using mdadm. It comes with Linux and is a form of software RAID. With that you only use up the equivalent of one disk for parity checking (parity and data are striped across all the disks so that if you lose a single disk you don't lose both the data and its parity bit so continue running). You can replace the failed disk and it will rebuild it for you to restore your redundancy.

If you create a RAID5 LUN (logical unit) then you can subdivide it with fdisk for partitions or put it in under LVM control.

Right, I'm glad that I asked the question. Losing a portion would be a fair less painful then losing ALL of it. I'm going to pick up another 1.5TB soon and I'm going to start undoing this LVM.

Instead I will make use of unionfs or something similar since the only reason why I went the LVM route is so that I could have all the data in one folder.


Oh, I know I'm screwed if a drive fails.. But it is a matter of *how screwed* hehe. I wont cry myself to sleep if I lose one drive... but losing all of them would hurt. haha

The raid 5 *is* a good idea... but again, losing this data wouldn't be the end of the world, I can get this data back, it would just mean re-ripping my dvd collection, family videos and such. And I'm running out of room in my case. Eventually I will need to look into a serious data storage solution and some rackmount cases. I have seen some interesting stuff on low cost SANs but that is for another day.

Thanks all!

Till then though... Separate and individual drives it is!

-->Woa<--. Parity checking came up in my futzing around the net, and it made me realize what you were talking about here.

I am definitely going to be checking this out more. I do not mind having one backup drive at all. 1 drive for ~12 drives is a pretty sweet deal. I realize that there is higher risk involved, since conceivably 2+ drives of 12 could fail at once... but.. unlikely unless it is from a catastrophically dieing PSU, natural disaster, theft etc.

I'm going to start doing some research on this, but if you have any good tutorials I'd be happy to see them.


This combined with realizing that my current setup is like having one big drive is a big eye opener for me. Thanks a bunch!!!

With that many drives you might want to consider spreading the risk some by creating two separate RAID 5 groups of 6 disks each. You'll end up using the equivalent of 1 drive in each for parity but you'll still have the equivalent of 5 drives capacity in each. Then losing 2 drives (if they're both in the same RAID group) will only lose half your data instead of all of it.

Note that losing 2 drives does happen on occasion. I've seen it here but it is less likely than losing just 1 drive.

It is important you set yourself some sort of monitoring so you'll know if you do lose one drive. I've seen more than one place where a drive died and no one noticed because everything kept running. Later when another drive dies everything goes away because it is the 2nd drive.

You might want to look at the options here http://en.wikipedia.org/wiki/RAID. With that many disks, you might want to go with RAID6 (survives 2 failed disks).
The key in any case (eg RAID 5 or 6 or ...) is to make sure your system notifies you of a dead disk before you run out of working disks.