we run several different Netbackup servers on Solaris/Windows/Linux.
1. Transfering you License has to be done via sales representative. Not sure if they are will do.
2. If you are using the same features i do not expect any additional cost for converting to another OS
3. there are some projects for filesystem compression (e2compr), but i do not know if they are in a stable state
4. I/O situation for what exactly? I/O in Linux is from my experience as good as the underlying storage devices (SAN/FC/NFS etc) and the drivers (Powerpath, Veritas ASL etc), so from my expectation there should be no problem if all is fully supported. Calculate exactly your requirements and size your system with an extra of 20% to prevent from problems.
5. virtualization is a personal decition if you like it you can do it, but be aware that every extra layer may slow down operations, we only use dedicated hardware for NBU
What cluster are you running? VCS or SUN Cluster?