It depends on the distro. I don't use debian or debian based distros that much but IIRC you make changes to the modules you want loaded on boot by editing /etc/modules. Actually, the process is more complicated than that in debian. There is a directory called /etc/modutils that contains a lot of module configuration files. A tool called "update-modules" reads the files it finds in the /etc/modutils directory tree and generates a configuration file called /etc/modules.conf which you are not suppose to edit directly.
Rpm based distros list the modules to be loaded in /etc/modprobe.conf or /etc/modprobe.preload for 2.6 kernels. Slackware and BSD have a differenet system whereby changes to modules to be loaded are done by directly editing various init scripts like /etc/rc.d/rc.modules.
Actually, the module loading process in all distros is heavily done by various init scripts with edits to files like /etc/modules, /etc/modproble.conf or /etc/modules.conf serving as ways to further tweak that process without directly editing the init scripts. Slackware has a module-blacklist script; anything you list in there will be banned from loading despite whatever is to the contrary in other init scripts. I don't believe debian has anything like that.
I'm also somewhat surprised that your rmmod command resulted in the module not loading permanently; that's not what usually happens. It could be distro specific or a debian thing, but something must have read which modules were loaded and made the appropriate changes in /etc/modutils to load only those modules on reboot. If you want to experiment, you can reload your onboard sound module with:
# modprobe <module name>
Reboot and see if it comes back and then rmmod it and see if it sticks on reboot.