Exactly as I suspected- you put the pam_tally2.so line at the end of the auth stack AFTER the lines:
auth sufficient pam_unix.so nullok try_first_pass
auth requisite pam_succeed_if.so uid >= 500 quiet
The "auth sufficient" line basically means "if the user types the correct password (pam_unix) then let them in without doing any further checks in the PAM stack." If that module succeeds, then pam_tally2 will never be called in your current set-up. You can fix this by putting the pam_tally2 requirement before
the pam_unix check. You would need to make a similar change for the account stack. I'd suggest consulting PAM documentation
since PAM can behave in sometimes strange and conter-intuitive ways as you've seen.
Note that simply changing "pam_unix" from "sufficient" to "required" will lock everyone out of your box. The reason for this is the requisite "pam_deny" call in the stack, which always refuses authentication. Because pam_unix is set as "sufficient", this call is skipped when the user types in a successful password. Another way of solving this, I think, would be to change pam_unix from "sufficient" to "required", move the pam_tally2 call directly above the pam_deny call, and change pam_tally2 from "required" to "sufficient".
As a final note, do be aware of the comment in /etc/pam.d/password-auth -- if you run authconfig on your system, the content of the file do get over-written by Red Hat's ever-so-helpful utilities.