whereis <filename> will tell you where the binary is. It varies a bit from distro to distro, but in general basic system binaries are in /bin (example: ls, passwd, chmod, chown, ect). Basic system binaries usually not needed by general users are in /sbin (example: mount, fdisk, fsck, ect).
Now, there are bin and sbin directories in /usr, /usr/local, and sometimes opt. Different distributions are setup in different ways, but most binary executables get put in one of those three places. If a program has global configuration files for all users thne /usr/local/share and /usr/share are good places to look for them (usually in a sub directory named for the program). Otherwise the configuration will be stored in a file starting with a . in your home directory. These files can be seen by adding the -a option to ls.