Disk/physical size more than virtual size

toshko3 · 07-15-2012, 01:07 PM

Hi I have a problem with qcow2 image files.
When the guest OS writes files it is all OK and the file disk size is increasing. When I delete something, the disk size is not decreasing (which I wrote is by design, for now). But, obviously, when I write again something it takes new "bytes" in the disk size. So after some time the result of the root image file of the virtual machine is like this:

Code:

qemu-img info ubuntu-pdc-vda.img 
image: ubuntu-pdc-vda.img
file format: qcow2
virtual size: 10G (10737418240 bytes)
disk size: 14G
cluster_size: 65536

and a confirmation:

Code:

du -sh ubuntu-pdc-vda.img 
15G     ubuntu-pdc-vda.img

It is the same with other images, too. I have big problems with "out of space" on the host machine, because their maximum sizes are meant to be what I have set in "virtual size". But they just rise and rise with no limit over time. I suspect a major bug here. Please, help!

dyasny · 07-15-2012, 01:28 PM

why do you think the image file size should decrease when you delete files in the VM?

jefro · 07-15-2012, 03:28 PM

I don't know enough about qemu-img info to say. What is the size of the drive you originally made? Was it the 10G?

toshko3 · 07-16-2012, 02:10 AM

Quote:

Originally Posted by dyasny

why do you think the image file size should decrease when you delete files in the VM?

I think it is reasonable. But anyway, I wrote that this is not the case. The problem is that the real file size of the image on the host is 15GB, but the maximum/virtual size I have set is 10GB. So more and more often there are out of space problems on the host. It is growing progressively!
The original file size was 10GB (with fallocate).

dyasny · 07-16-2012, 03:28 AM

ok, when you create a qcow2 image with qemu-img you set the max size it will reach. It should not be able to grow beyond that size.
Having said that, the size will also not decrease automatically when you delete files, because when you delete a file, the data remains and keeps taking place. Only the FS becomes aware that this space can now be reused by other files, so if a new file is created it can overwrite the actual data in the space that was occupied by the deleted file, but until then, the space is taken. the outside layer is not aware of the in-guest FS changes, for the qcow2 image and qemu this is just blocks of storage space, that the VM writes to or reads from. Whether or not they are actually allocated to existing files or are occupied by data that is no longer belonging to a file doesn't matter. This is why the image will not decrease in size automatically. There are ways to reclaim that space, but they require some extra steps. This is, btw, the case with all the types of virtual images, whether it's RAW, QCOW2, LVM, VMDK, VDI, VHD... you name it, the principle remains the same.

As for the image becoming larger than it was allocated to be, it might be a bug (are you using ubuntu by any chance?), but also first I'd like to know how the image was originally created (what commands were used)

toshko3 · 07-16-2012, 04:37 AM

Yes, didn't think about how the filesystem works and you reminded me now. You are right! :-)
Yes, I am using Ubuntu server 12.04 64bit. (latest updates), ext4 filesystem on both guest and host. The command, I created this image is this:

Code:

qemu-img create -f qcow2 -o preallocation=metadata ubuntu-pdc-vda.img 10737418240 (not sure about the exact bytes, but around this)

After that, in order to get the size in bytes for the next command:

Code:

ls -l ubuntu-pdc-vda.img

After that allocated the space for the image for better performance:

Code:

fallocate -l theSizeInBytesFromAbove ubuntu-pdc-vda.img

Thanks!

dyasny · 07-16-2012, 11:19 AM

Should have known it's ubuntu...

Anyway, lets try something simple

1. qemu-img create -f qcow2 image.img 10G
2. ls -lsh image.img
3. start a VM on this image, write data, see if you can grow it beyond the 10G mark, by repeating the ls -lsh command

toshko3 · 07-17-2012, 07:40 AM

It seems that this behavior is not present at the beginning. But at the moment we create a snapshot it is starting to write on top of the current size. So, it is like this:
1. Image is:

Code:

qemu-img info WindowsXP.img 
image: WindowsXP.img
file format: qcow2
virtual size: 15G (15728640000 bytes)
disk size: 13G
cluster_size: 65536

I write some files and it doesn't become more than that.
2. I delete some files, then write again and it doesn't changes size.
3. I create a snapshot:

Code:

qemu-img info WindowsXP.img 
image: WindowsXP.img
file format: qcow2
virtual size: 15G (15728640000 bytes)
disk size: 13G
cluster_size: 65536
Snapshot list:
ID        TAG                 VM SIZE                DATE       VM CLOCK
1         test                      0 2012-07-17 09:52:25   00:00:00.000

4. I write something with a size of 587MB and it becomes larger (with 587MB)

Code:

du -sm WindowsXP.img 
14102   WindowsXP.img

5. I delete it and then write it again. It becomes more larger... (again with another 587MB)

Code:

du -sm WindowsXP.img 
14703   WindowsXP.img

6. I delete it and then write it again. It doesn't change this time.
7. I write a copy of it. It becomes larger (with 587MB)

Code:

du -sm WindowsXP.img 
15309   WindowsXP.img

8. I delete it and then write it again. It doesn't change this time.
9. I write a copy of it. It becomes larger again (with 587MB)

Code:

du -sm WindowsXP.img 
16010   WindowsXP.img

10. I write another copy of it and it stays the same.

Code:

du -sm WindowsXP.img 
16010   WindowsXP.img

11. I write another copy of it and becomes larger again.

Code:

du -sm WindowsXP.img 
16913   WindowsXP.img

Code:

qemu-img info WindowsXP.img 
image: WindowsXP.img
file format: qcow2
virtual size: 15G (15728640000 bytes)
disk size: 17G
cluster_size: 65536
Snapshot list:
ID        TAG                 VM SIZE                DATE       VM CLOCK
1         test                      0 2012-07-17 09:52:25   00:00:00.000

12. Create a snapshot.
13. I write a copy of it. It becomes larger again (with 587MB)

Code:

du -sm WindowsXP.img 
17572   WindowsXP.img

14. I delete both of the snapshots.
15. I delete the file (in guest with the size of 587MB) and write it again. No change in size.
16. I delete the file again and write it again. No change in size.
17. Create a snapshot.
18. I delete the file again and write it again. No change in size.
19. Delete the file.
20. Create a snapshot.
21. Write the file again. No change in size.

Well from all this, I can conclude that most probably:
1. The problem occurs only when there is an internal snapshot present.
2. The problem is not "by design" because the behavior is not consistent (for example, 13. and 21. should be with the same result, but they arent't)..

At the end of the day, after these procedures (4 creations of snapshots, 2 deletions and some writing and deleting of internal guest files) the result is this:

Code:

qemu-img info WindowsXP.img 
image: WindowsXP.img
file format: qcow2
virtual size: 15G (15728640000 bytes)
disk size: 17G
cluster_size: 65536
Snapshot list:
ID        TAG                 VM SIZE                DATE       VM CLOCK
1         test2                     0 2012-07-17 14:55:17   00:00:00.000
2         test3                     0 2012-07-17 15:27:43   00:00:00.000

and

Code:

ls -lsh WindowsXP.img 
18G -rw------- 1 libvirt-qemu kvm 18G Jul 17 15:43 WindowsXP.img

dyasny · 07-17-2012, 08:36 AM

OK, I guess I'll just need to explain what a snapshot is then.

When you create an image, say with 10G in size, it can fill up to 10G. Lets say we did fill it out, and you have a disk image in a certain state, that takes 10G of disk space.
Now you take a snapshot, the snapshot is actually a new image that takes zero space, and can grow up to 10G again. When you write to the snapshot, it can fill up to 10G, so all together, you will be wasting 20G for both snapshots if you fill out the second snapshot. The VM will still see only 10G, but on the storage you'll be taking 20G already. The longer the snapshot chain, the more space you will actually be using.

This is by design and isn't a problem, if the previous snapshot didn't hold all the data, (10G in this case), you would not be able to revert back to the state it holds.

toshko3 · 07-17-2012, 10:20 AM

Then why on step 21. there is no change in size?

dyasny · 07-17-2012, 02:52 PM

how do you create the snapshots?
can you post ls -lsh after every action, along with qemu-img info ?

toshko3 · 07-18-2012, 01:56 AM

The snapshots are created like this:

Code:

virsh shutdown WindowsXP

or shutting down from Windows itself.
After the machine is down:

Code:

qemu-img snapshot -c test1 WindowsXP.img

After the snapshot is created and verified with "qemu-img info WindowsXP.img":

Code:

virsh start WindowsXP

and then proceed with the copying of files in Windows itself, from the network.
I cannot post a ls -lsh after every action, because yesterday it took me all day to make the test. But, I did it at every step and the info I'm saying is right, believe me! I just didn't post it where there is no difference, so the reading of my whole post would be easier.

mastifo · 12-18-2012, 03:16 AM

Any solution to that Problem now? How can i control the steady growth of my disk-size?