hmm, sorry for the LONG wait between replies, but just in case anyone cares, here's something you might try:
First, upgrade to mplayer 1.0pre3 if you haven't already.
mencoder dvd://1 -oac mp3lame -lameopts vbr=3:br=128:q=0 -ovc lavc \
cbp:mv0:vbitrate=880 -o file.avi
Should turn out fairly well. I consider 880 to be a little on the low side so I would also use -vf hqdn3d=6:4:7 (or so) as well. Finally, you save some bitrate-per-quality by cropping off black borders from the top and bottom, then rescaling so the height is still a multiple of 16. Use mplayer -vf cropdetect to help with this.
If it really matters, you can experiment and tweak a bit more. Good candidates for experimentation include the hqdn3d parameters (higher numbers make the video more compressible but also add blurriness and, for the third option, ghosting); playing with vmax_b_frames (vmax_b_frames=1 or 2 might sometimes be better); adding "predia=2:dia=2" to lavcopts (sometimes 3 is good); and possibly using the lumi_mask and dark_mask in lavcopts. These last two are debatable; they can make some parts more compressible but the loss in image quality often isn't worth it (making the image either too washed out at times, or too dark at times). It all depends on the source material! For certain unusual source material, much different options might make more sense.