The agent is only used in establishing the initial connection. Specifically it is involved in the authentication. So the disconnection is happening for some other reason, unless the initial connection is not actually happening in the time alloted. You can throw a -v in there temporarily to increase the amount of information it shows you.
Since you are copying the same file again and again, it looks like
rsync is the way to go. It actually runs over SSH these days so it can use keys and the agent. If you need to specify some weird options or specific keys then that can be passed to
rsync
Code:
rsync -e 'ssh -i ~/.ssh/some_key_rsa' -a -v file user@server:path/
You can also make the configuration options permanent by putting them in a
~/.ssh/config entry for the target host.
Code:
Host server
Hostname server.example.com
User user
IdentityFile ~/.ssh/some_key_rsa
ConnectTimeout 30