I'm happy if it worked.
Actually, I'm not sure that both value (VertRefresh & HorizSync) are correct for your monitor.
You have to find and then set it to the correct value to do better.
For 1152x864 resolution, you can try to edit xorg.conf again and then add 1152x864 at modes row:
for example:
SubSection "Display"
Depth 24
Modes "1152x864" "1024x768" "800x600" "640x480"
EndSubSection