I think that the time will quickly come, either when you can't log-in any more, or when your "secret" paper list of passwords gets pick-pocketed and photocopied.
I would, first of all, bring a computer with me and set it up with password-protected (encrypted...) VPN certificates
to allow access to the target servers. Access to the servers e.g. by SSH or other means is not possible, except through the VPN tunnel, and access to the tunnel is only possible for a bearer of a currently valid certificate (i.e. you) who knows the encryption key for the same and is thereby able to use it.
SSH logins, using once again digital certificates (encrypted, again) is stronger in my opinion than any password system. And, this SSH cannot directly log-in to root.
It must be used to log-in to a maintenance account which might be the only member of the "wheel" group or otherwise via PAM the only one who can from there
gain access to root. This maintenance account has read/write access to directories and files of elevated concern, such that full root access is not required to get to those files. (The maintenance account is super but not super-duper.)
So, a keylogger wouldn't do any good: you have to steal your laptop, hold a gun to your head to get the encryption key for the certificate, and so on . . .