It depends on how you're installing the package. If you're using a distro that has a package manager (e.g., yum or apt) and installing prebuilt binaries, things are going to go where the software has been compiled to put it. Since everything is compiled for you, you don't have much choice. Debian falls in this category.
If you use a source-based distro like Gentoo, everything (not really) is built from source on your machine using scripts. You could hack the script and put things where you want them. A package manager is still being used, but the default install locations are chosen by someone other than you.
You could use something like (C)LFS where there is no package manager unless you install one or create one. In this case, you simply pass --prefix=<whatever> to the configure script to install the package in <whatever>. You can usually provide finer control with other options as well (e.g., --libdir=, --sysconfdir=, etc.). This paragraph also applies if you install a package outside of your package manager regardless of the distro.
That said, putting libraries, binaries, *.pc files, etc. in odd locations is likely to lead to a borked system. At a minimum, it will become more cumbersome for you to administer. There is a
standard that is adhered to by the distros to varying degrees, including not at all. Stick close to this and your box will be easier to administer.
BTW, you'd be surprised what might use a library. You're right the pingus libs may not be used by anything else, but the standard has a place for things like that beside /usr/lib.
