It depends on what is starting it.
(some more likely than others)
It could also be your window manager or other things. Are you sure that it's xscreensaver and not just a DPMS setting in X blanking the screen after a certain amount of time? Well rereading the OP, the password would likely be the screen saver. But it could also be the display manager (kdm, gdm, wdm, xdm.....)