The proper way to do it is to get udev to set correct permissions, but I made a small script that can change permissions of the scanner to allow anyone in scanner group access, rather than a+rw which makes it readable/writable to all.
http://www.unrealize.co.uk/scripts/sane/
There is a readme and the script file 'scanperms' which you can source from rc.local for example.
Also, I notice your groups and I would say it would be beneficial to add yourself to some more groups than just users, plugdev and scanner. E.G.:
users tty floppy audio video cdrom plugdev power netdev scanner
But especially power, audio, video, netdev (for wicd) and cdrom.