Best LVM2 volume group extent size for photo storage

TheOneKEA · 03-08-2010, 08:29 AM

I just finished setting up an LVM2 volume group with 128 MB extents, for the purposes of running a series of RAID 1-style LVM2 mirrored logical volumes. I was doing research online with Google and noted that increasing the extent size from the default of 4 MB works well for volume groups containing logical volumes with large files.

My camera currently shoots in RAW+JPEG, and since I set up the logical volume for the purpose of storing my photo collection (currently at 11 GB) I have dozens of 2 MB and 11 MB files in the logical volume (formatted with ext4). Is a volume group extent size of 128 MB too big/inefficient? Should I decrease it, or does the use of ext4 offset the large extent size?

Physical volumes:

Code:

  PV         VG              Fmt  Attr PSize   PFree   DevSize                            
  /dev/sda10 RAID_One_Mirror lvm2 a-   640.00M 512.00M 651.04M
  /dev/sdb   RAID_One_Mirror lvm2 a-   298.00G 234.00G 298.09G
  /dev/sdc   RAID_One_Mirror lvm2 a-   298.00G 234.00G 298.09G

Volume groups:

Code:

  VG              Attr   Ext     #PV #LV #SN VSize   VFree                               
  RAID_One_Mirror wz--n- 128.00M   3   1   0 596.62G 468.50G

Logical volumes:

Code:

  LV       VG              #Seg Attr   LSize  Maj Min KMaj KMin Origin Snap%  Move Copy%  Log           Convert
  Pictures RAID_One_Mirror    1 mwi-ao 64.00G  -1  -1 254  3                       100.00 Pictures_mlog

Relevant commands:

Code:

pvcreate --metadatacopies 2 /dev/sdb
pvcreate --metadatacopies 2 /dev/sdc
pvcreate --metadatacopies 2 /dev/sda10
vgcreate --addtag RAID1 --physicalextentsize 128 RAID_One_Mirror /dev/sda10 /dev/sdb /dev/sdc
lvcreate -L 64G -n Pictures -m 1 RAID_One_Mirror
mke2fs -L Pictures -m 0 -O large_file,dir_index,extent,uninit_bg,sparse_super -t ext4 /dev/mapper/RAID_One_Mirror-Pictures

sirmonkey · 03-02-2012, 07:56 PM

Two year bump... because this post comes up on alot of google searches...

what is the answer? I have no idea but I would like to know.

ba.page · 03-06-2012, 02:20 PM

I don't believe the file size matters in choosing an extent size. you base your filesystem type on the size and volume of files you plan to use. ext4 will do fine for a ton of small files (yes, 11mb is small).

the physical extent (PE) matters in a different way.
A logical volume (LV) is limited to 65,536 PE.
so choosing a PE of 4MB (the default size in current kernels) limits your LV to 256GB.
choosing a PE of 8MB will yield a maximum LV of 512GB.
(and so on)

the crappy part:
you can't alter PE size without fremoving the LV first, and rebuilding the physical volume with a larger extent size.