[SOLVED] .vdi transparent compression btrfs/zfs

qrange · 11-19-2019, 04:56 AM

is there any point in using transparent compression on very large files that are frequently modified such as .vdi images?
I've read that btrfs compresses file-by-file not by blocks, so vbox would become very slow if vdi is compressed?

I'd like to create compressed writable filesystem partition just for .vdi

thanks.

Firerat · 11-19-2019, 10:52 PM

with zfs, lz4 compression

a 21G .qcow2 reported by ( du -h ) as 6.7G
in the VM, df -h / reports 8.6G ( and ncdu / as 8.5G )

I've not benchmarked performance, but I wouldn't describe it as slow

vdi is also sparse?
so I would expect similar compression
which is of course dependent on the actual data

it has been a while since I use btrfs, I think I did turn off compression for images, not sure it made much difference

I do remember you had to make a copy after changing per file attr
or some such similar juggling.
I think I had to set attr of the dir, move file out and then copy back in and delete the "original" file

qrange · 11-21-2019, 02:11 AM

thanks.
vdi is dynamic.
I've used btrfs with zstd and it seems to work fine, 50% is compressible with force option:

Code:

compsize win7-64.vdi 
Type       Perc     Disk Usage   Uncompressed Referenced  
TOTAL       59%       13G          23G          23G       
none       100%       11G          11G          11G       
zstd        20%      2.4G          11G          11G

btrfs wiki says:
"Compression does not work with direct IO (DIO), does work with COW (the default) and does not work for NOCOW files. If a file is opened in DIO mode, it will fall back to buffered IO."

I think vbox uses DIO. I've enabled 'host-caching', it might matter.

qrange · 11-22-2019, 01:40 AM

haven't noticed any slowdown, defragmenting the vdi might help.

deduplication should also help if we have several .vdi with same OS. have you tried it?