If you look at top while this is happening (slow response in KMail) what do you see? Is KMail using a lot of CPU time? If you can get something like an LMSensors client going (I use GKRell) you should be able to access a graphical representation of how "busy" your network interface is - is there lots of traffic going back and forth? Conjecture: If you have a DMA issue and a fast (real fast?) broadband connection, and DMA is not active in your kernel, you -might- run into latency as the CPU has to be involved in handling data transfer that could have been done without CPU intervention via DMA?
It is definitely not like this for me - no matter if I am "sending and receiving" mail under KMail in FC3 on a 184.108.40.206 kernel, it stays very response. Granted, I have an older KDE (3.3) so my KMail version probably isn't up to date.
I did have this type of problem though on an older machine, but that was because DMA was not active in the kernel, and all programs got slow if IO was happening (as might happen with you if your LAN his the gas). What else are you running simultaneously that might eat CPU time?
I assume you do eventually get your mail, and once it is in the local inbox KMail is responsive again?