Video generally needs to be in either mpeg1 or mpeg2 format to be playable on hardware DVD players*. Not only that, but it has to fit within certain parameters of size, framerate, bitrate, and such. And the same goes for the audio format.
Wikipedia has the details if you're interested.
Also, most DVD players cannot handle "raw" video; that is, just burning an .mpg to disk is not usually enough to allow playback. They have to be "authored" with an appropriate menu structure (though the don't necessarily have to have menus) that gives the player all the details about the video and how it should be played, as well as defining chapter breaks and other fancy effects.
So what you need to do is 1) convert your videos to compliant mpeg format, and 2) use an authoring program to create the menu structure. Then you can burn the completed structure to a blank disk.
For conversion, I recommend avidemux. It's a fairly simple, but powerful program to use. Simply open up the video you want to convert and choose DVD from the video format on the left. In the configuration panel, you can vary the bitrate, which may let you fit more video on the disk, but at the expense of picture quality. And in the filters dialog, be sure to punch the dvd resize button at the top. In the audio field, choose mp2 or ac3, and be sure to resample the audio to 48000 in the filters. Finally, in the format box choose MPEG-PS, then just click save to create your finished .mpg.
For authoring, I suggest using dvdstyler. It's the easiest and most stable program I've tried so far. You can easily create a simple menu system with it. The documentation is good and explains a lot about how the DVD system works. I suggest you read it all. Two hints: be sure to add a vmgm menu, because some DVD players and programs don't like disks without one, and tell dvdstyler to disable jumppad, because it's an unnecessary hack that causes more problems than it solves.
When done, you should have a folder called VIDEO_TS, which you can burn to DVD using your favorite burning program (dvdstyler also has an option to burn directly to disk). Be sure to use the "DVD video" option though, to ensure that the structure is burned to disk properly. If everything goes as planned, you should now be able to play the disk from any DVD player. You might want to use a rewritable disk for testing first though, to save on the cost of burned coasters.
Good luck and happy DVD-ing.
*There do exist players that are capable of playing non-standard formats, such as divx video or mp3 audio, or that can play unauthored video. But disks using such formats are not guaranteed to play everywhere. It's usually best to stick with the standard DVD format.
