I figured out the problem.
1. A recent change to the PAM configuration changed the password policy. This change required all users to change their password on next log-in.
2. My user profile was added to the system incorrectly in July when I started working here (still trying to figure out how and fix it). When I change my password, it only updates my Kerberos password and not my local password on each node. When anyone else changes their password, it changes their Kerberos password and their local password on all nodes (password synchronization across the entire cluster). I have to log onto each of the 9 nodes and change my local password independently from my Kerberos pw (still troubleshooting).
3. The problem in step 2 was hard to figure out because we must ssh to these classified systems. Using ssh causes a different log-on behavior in that once you change your password, it logs you out and makes you log back in. Since my pw updated only my Kerberos password, the system kept prompting me to change the pw...of course Kerberos would only let me do a pw change once a day.
To fix this I had to:
1. Gain access to the room that contained these systems so that I could log in at the console instead of ssh. This is to prevent me from being logged out after the initial password change.
2. Log-in using my Kerberos password.
3. Change my local password using the passwd command so that it matches my Kerberos password. (My profile only)
4. Repeat steps 1-3 on all nine nodes in the cluster (my profile only). This is because I still haven't fixed the problem with my profile that's preventing synchronization of my Kerberos and local passwords across the entire cluster.
5. Have each user ssh in and change their pw. Others only had to do this once (any node) and the entire cluster is updated properly.
Last edited by Enochs; 12-13-2012 at 04:58 PM.