As far as I know, a source package is one you compile yourself and a binary is precompiled. So binaries can't be adjusted if you've got a weird system or optimized for your system. With source, you still might have problems if essential things are missing, but it's more flexible and can be tailored to your system. Plus, if you're a programmer, you've got the code to hack. Even as a non-programmer, you can try to do some configuring for install.
Basically, binaries are ready to go and source takes more work for better results. Source results in a binary; binary came from source.
At least, that's my understanding.