The ogg format (or more accurately the
vorbis codec,
ogg is just the container format) is generally considered slightly superior to
mp3 in quality. That is, you can get slightly better to-the-ear sound for the same file size, or conversely the same quality sound in a smaller file. On the other hand, ogg doesn't have the same level of overall support that mp3 does; in particular most portable music players cannot handle ogg. Most of the better software players have support though (but don't expect Windows Media Player to recognize it!).
And as already mentioned, IP issues affect mp3, meaning that most FREE distributions cannot include the codec by default. It is legal for individual users to install mp3 software on their own systems though, and codecs are available gratis.
The ogg container and vorbis codecs, OTOH, were created specifically to provide for a truly FREE alternative to IP-encumbered multimedia formats. There are no known issues involving patents or other IP (although you never know when it comes to software patents), and the code is all open source. Anyone can distribute them.
So what this boils down to is, if you want superior quality and freedom, use ogg vorbis, if you need portability and don't mind the restrictions (admittedly non-burdensome at an individual level), then mp3 will do. But you'll probably have to install it yourself.