It's hard to guess without knowing exactly what are you comparing against... That would help. What are you used to?
You also are mixing apples and oranges here.
You can perfectly develop an OS based for a microcontroller in any Linux (binary or not), or even in Windows or MacOSX if that's your call. There are also virtual machines, native sdks and that stuff for that.
And you can always cross-compile, of course.
So, what are you exactly trying to do?
Are you maybe developing apps for a mobile phone? Are you developing or hacking an OS for an embedded device? Developing regular applications? Developing drivers? We need to know that.
Now, about source based distributions, what I use and know is Gentoo. This OS is compiled from the Ground, and you have all the source code right there. But that really shouldn't be a thing that will stop you from using your favourite binary distro either. There are -lib and -devel packages in most distros that you can use to get the source of the libraries or applications that you wish, so you can #include them in your projects or something... we would need to know more specific details about "WHY" you need (or why you think you need) the sources of your whole OS.
You can always get them, but maybe you are trying to kill a fly with a bomb here.
About your other question, you really shouldn't need to rebuild most parts of anything nowadays. That's what build systems
are for. There's gnu make/autotools/automake, cmake, pmake, nmake, etc. Those should evaluate the dependencies, modification dates et al, and decide what parts of given project need to be rebuilt. All the frontends use this nowadays.
Definitely, you don't need to rebuild your OS each time you compile a program.
Source distros, however, need a way to control ABI breakage. Sometimes, when you update a given lib (let's say SDL or libpng), all or most of the programs that are linked dynamically to that library will remain broken (and won't work) until you recompile them against the new version of the library.
In Gentoo, portage (it's "package manager", to call it something) can take care of that. There's also a now to be deprecated tool to do that, that's called "revdep-rebuild". In Linux From Scratch the one to take over that responsibility would be the user.
In those cases, just like you have to rebuild the OS applications depending on libpng, you would also need to rebuild your program if it depends on libpng.
But I have no idea if that's what you are talking about, since you spoke in such general terms...
I hope this offer at least a starting point to iron the issue a bit. Feel free to ask for details if you need more info.