"Extract" is something usually done with a .tar.gz file. Then, ./configure, make, and make install usually come next.
Binary rpms are usuall installed with 'rpm -i'. If you have an earlier version of VLC installed, rpm -U should work (it would upgrade the existing installation). If you do not have VLC installed (from an rpm), then rpm -U will not work; you will have to use rpm -i to install.
In order to give you better info, post the name of the package you want to install, and the exact command you are using which fails to install the package.