Programmers write programs in source code. C, Java, etc. Unfortunatly, computers actually can not read this code, its only "human readable".
Computers speak in machine code (binary, 0s and 1s). So a binary is just that. A program where the source code has been converted ("compiled" is the usual term) into machine ready code.
When you compile (./configure, make) something, all you are doing is converting source --> binary. So a binary makes it easier for you.