The problem isn't with consolegroups if audio groups is listed when you use the groups command. I'll try to explain it better anyway
Consolegroups is a list of groups that the users of the system get membership to when they login from the text mode (from the console). If you use xdm instead of text-mode login, consolegroups has no effect. This is what Slackware's login.defs says about it:
# List of groups to add to the user's supplementary group set
# when logging in on the console (as determined by the CONSOLE
# setting). Default is none.
Oh, and by the way it seems to be CONSOLE_GROUPS, not CONSOLEGROUPS like I told you earlier, sorry about that. Manpage of login.defs also reveals that if you are using PAM (Pluggable Authentication Modules), the whole login.defs does nothing. I'm not sure if this is the case with Gentoo. PAM to my understanding is not without problems, and it's quite possible that your audio problems are related to it. However, since I don't use PAM myself, I'm unable to help you with it.
When it comes to xdm, it really should be used by changing the runlevel of the system, that's the way it's meant to work. It's real easy too, just look into your /etc/inittab file and find a line that says:
Then change the number 3 on that line to whatever your X runlevel is (probably 4 or 5, help is usually available in inittab file). Worth a shot anyway.
You can also set the runlevel when booting the system if you want to test it before editing inittab. Type linux 4 (or linux 5) at the boot prompt.