Check out
tovid, a script I wrote for converting video to DVD and other formats. I've found mencoder itself to be somewhat flaky for creating mpeg2 video (which DVDs must be), so my script calls upon utilities in the
mjpegtools package.
As for choosing the right bitrate, from what I can tell the mjpegtools mpeg2 encoder uses a variable bitrate, up to the limit you specify. In my script, there's a constant you can set for the maximum bitrate, but I've found most video to fall below about 7500kbps (the default bitrate for mpeg2enc). At that bitrate, you're guaranteed to fit about 81 minutes of video and audio on a 4.7GB DVD, but in my experience a more realistic average is around 240 minutes (four hours!) I'm not sure what commercial DVDs use, if there is even a standard bitrate; I know those 'superbit' DVDs you see have a higher bitrate, achieved by sacrificing the extra space usually spent on commentaries and deleted scenes. I wouldn't be surprised if most commercial DVDs were around 7500kbps.
Also, in using my script, note that there are a couple additional formats supported by DVD that can help you get a lot more onto a single disc: 352x240 (Video CD resolution) and 352x480 (half-D1 resolution) are both supported by the DVD standard, as far as I can tell; if you have stuff taped from TV or low-quality movies, those resolutions might be better suited than the 720x480 standard DVD resolution, and could help you cut down on those bitrates (less resolution, less bitrate needed). I encoded a TV series to half-D1, and managed to get about 350 minutes per disc at 3600kbps, enough for seven or eight 45-minute episodes per DVD.
Good luck! Let me know if you have any problems with the script.