Check out ccsm (compizconfig-settings-manager), there are settings aplenty there

I believe it is the most comprehensive settings editor for compiz besides editing its config files by hand. (In fact some have even seen the need to simplify it abit, and hence we now also have simple-ccsm)
As for your question: In ccsm go to General Option, then to the tab named "Focus & Raise Behaviour"; uncheck the option "Click To Focus".... voila