I don't have Red Hut Linux Enterprise V5 and Dovecot IMA but nobody else has replied yet ...
Surely hitting a user's quota limit affects a service that is creating files for users! A well-programmed service would detect file creation error (for any reason, including quota limit) and take graceful action rather than hang or become unstable.
Regards the du/repquota discrepancy ...
Check the man page for your specific repquota. Most man pages say it reports in kilobytes, others do not specify the unit. The quota suite works in "blocks". Each file system is formatted with a particular blocksize, usually 1 kB. I don't know if the quota system works in fixed-size (1 kB) blocks or in the block size of each file system. What is the block size on the /dec/sda9 file system?
Please confirm that you running [root@kin ccc]# du -a <username>
in a directory on /dev/sda9 that contains a directory called <username> which contains only
files belonging to <username>, including any hidden (.*) files?
Are you absolutely sure that <username> has no other files on the /dev/sda9 file system?
What does du report with the --apparent-size option?
Does Dovecot IMA create sparse files?