gunter's link is not a bad start, but it can be intimidating digging deeper into what a library is without a programming background. So, I'll try to supplement the information with a plain-english explanation.
A computer library is sort of like a toolbox. Programs that run on the system can use bits and pieces of the library to perform a task (much like you can use a screwdriver to open your computer case or tighten a loose doorknob). The library contains quite a few tools, and nearly every program will use at least one of them. The main benefit from doing this is a space savings. Those tools are shared between programs, and don't have to be repeated (and stored) in every program that uses them (again, just like you wouldn't need a screwdriver for everybody in your house; just one that can be shared).
If you ever want to upgrade the library, it gets sticky. The sharing which was once a good thing, becomes bad, because if the upgraded library is broken, many (possibly all) of the programs that use it will break. Secondly, if the upgraded library is significantly different, the tools might be stored at different locations (meaning you might end up with pliers instead of a screwdriver).
That's a really, really abstract view, but gives a basic understanding of what it is. If you forget everything else in this post, remember this: all the programs on your system depend on glibc; if glibc breaks, then you're up a proverbial creek.