first off I think transcode might use mencoder, then again I am not sure, if you are planning to do this in windows I cannot help you, if you are planning to do it in linux then I would need to know what distro you use. or if you have the developement tools installed on the linux system you can compile mplayer yourself, default cxonfiguration will work for this.
first go to a terminal and type mplayer
then also try mencoder
if eather work you have mplayer installed, if only the first works you will need to re-install
download the source from www.mplayerhq.hu
copy the tar.gz or tar.bz2 file into /usr/src (as root)
(still as root)
tar -zxvf file.tar.gz
tar -jxvf file.tar.bz2
depending on the file name
use cd to change intot he directory it created
type the following
./configure && make && make install
when that finishes (might take a bit) try running mencoder, if it works you are good to go, read the following to rip the dvd:
it might look a bit daunting, but if you read it and not skim it you should be fine.
depending on your distro you might be able to find an rpm/tgz/deb of mplayer w/ mencoder and you will not need to compile it from source