I've gotten this working on a Debian installation. I wanted to be able to hear IM sounds while XMMS was playing. Just like you, I've experienced the sounds playing immediately after an mp3 stops. My guess is the sounds are being queued. The IM sounds playing over and over again are probably from multiple IM sounds being queued.
Following the hyperlink for the
DMIXPlugin page worked for me. Unfortunately, I don't know the details of exactly what's going. What worked for me was installing the necessary packages followed by putting the configuration code in Step #6 into my ~/.asoundrc.
I have a small write-up of what I did at
rayli.net.
By the way, what IM client do you use? In case you use Gaim, you might need to change the
Sounds >> Sound Method preference to
command. Then set the command to
aplay -q %s. I'm not sure if this is the best way to do this, and it worked for me.