Controlling access by group membership is one way of controlling access. You can remove a user from the "audio" group for example.
Using udev to control permissions of devices as they are created is another possibility. Udev is sometimes used to change a device's group ownership from the default root ownership.
A more general method allowing more granularity is to use Policy Kit. Look at the comments of /etc/polkit-default-privs.restrictive if you have it.
Code:
#
# /etc/polkit-default-privs.restrictive use in an envirenment where
# hosts are centrally administered and users should have minimal
# privileges. Privileged operations require authentication as admin.
#
# Please do not modify this file, use polkit-default-privs.local instead.
#
Also this manpage may help:
man 5 polkit-default-privs
If you have Polkit installed, it probably came with html documentation somewhere entered in /usr/share/doc/.
The udev/dbus/HAL/Polkit combination is fairly new, so I doubt that there is a PAM module to retrieve a group of policies from an LDAP directory when a person logs in. There are commands to change policies on the fly that would probably be used. ( Sounds like a good open source project if it doesn't exist )
Having restrictive policies by default, and removing select restrictions depending on the users credentials will probably do everything you want.
Sorry I can't point you to a particular project or HOWTO to implement it.
Oh, one more thing. Polkit tends to control access using ACLs instead of group membership. That is something you can manually do yourself if you wanted, but this would be unworkable for a large number of users or hosts.
