Xine should have support for those, so should MPlayer. MPlayer requires the Win32codec-all before it can install... so it should be able to do divx and all straight out of the install.
http://cambuca.ldhs.cetuc.puc-rio.br/xine/
I know that I followed that, and it worked just fine, there is a Jem Report as well on the subject. Both can get you to the goal you desire, though.