Are you using yum? Do you have the RPMforge repositories set up?
Code:
http://stanton-finley.net/fedora_core_5_installation_notes.html
For FC4, the x11vnc rpm is in the dries repository, which is part of the RPMforge partnership, so I would imagine the same would apply for FC5.
cheers,