nfs root extremely slow... ubuntu 16.04 server
 

With my experimentation with linux some days i create 5+ vms. As such it seemed logical to make a pxe boot server for myself. After much struggling it works great, except it is extremely slow. The TFTP files transfer over quickly but once it gets to actually reading the nfs "disk" the transfer speed is horrid.

In a test vm it took me almost 10 minutes to get through the first "retrieving" part of the ubuntu server installer. In theory it should be faster than a usb 2.0 flash drive, but not even close.

No idea where to start to diagnose and fix this. Appreciate suggestions and help to solve.

*EDIT* Tested on bare metal, same result on my laptop.
iperf test.

I think it will depend on the NFS server...

How many NFS server processes are available? The default is 8, but I have actually seen 4 as a default - for 5 VMs I would suggest 12.

How many CPUs on the NFS server (if less than 8 it might slow down a good bit), and the VM host (at least 1 for the host itself, preferably 2)?

Is locking used/supported?

What about the routing of the VM network to the NFSserver?

Are all 5 VMs active/booting simultaneously?

And since you are testing, is the NFS server also the host for the VMs?

If all 5 VMs are booting simultaneously then a 1GBit interface would be divided by 5 (6 to account for some additional overhead) which would come out at about 170MBit to 200Mbit throughput (or very roughly 20-25 MByte/sec I/O each - which would be VERY slow)

There is also the issue of what else may be active on the network.

Nfs threads at 8. Vm host is laptop, quad-core core i7. One vm at a time, still testing it. Nothing else to speak of on network, nothing high bandwidth intensity that is. The vm host is connected to server via gigabit switch. I have same problem on kvm vm on the server hosting the pxe server and nfs shares as well.

I do know that when I was mounting my iso nfs shares and installing from that on laptop virtualbox it was good and fast. Is something specific to the pxeserver.

Sloppy reply, lq.org not good on my phone :(

1Gbit/sec would translate to about 128MByte/sec transfer.

It shouldn't be the PXE server as once the kernel+initrd is loaded, the PXE server is out of it. If the other response on the laptop is normal, then it might be the virtual network connection to the VM.

Usually there is two levels of routing going on at that point - the VM virtual network to the host to the remote server. I don't offhand know the overhead that imposes as my VMs use local disk emulation just to see if things run. It is possible the VM doesn't have enough memory and is having to page over NFS (which is really really slow). I have 8 GB of memory and usually give between 1 and 2 GB to a VM for operation. I have tested less than 1GB, and it gets rather slow (sometimes can't even install).

I think that the only plausible explanation for this sort of "motionlessness" is some kind of timeout. Check the network logs carefully and be sure that you see a commensurate amount of network traffic taking place.

I'll look at the logs when I get home. I eliminated the vm question by test loading and installing on spare partition on my laptop bare metal. Took very long time.

Marking as solved. I looked over several guides and I found a few that were using a Kickstart file that directed the installer to get it's files from the nfs share instead of the internet. That is why it was so slow, my internet is mediocre at best and it was downloading everything instead of actually using the nfs share. adding a kickstart directing it to use the nfs share solved the issue.

That will do it :)

Ah... Never thought about it doing a network install. That would be slow.