I've heard nobody complain about the driver.
What everyone complains about is locales, settings. Basically there's a whole mess of stuff linking character sets with locales(local set up stuff for where you live), and then some bright spark came along with unicode, and that went in with extreme difficulty. There's also Xmodmap, xev, and other ways of getting the character you want on the key you want.
The basic problem is that from the 256 possible combinations, you want all the latin, arabic, cyrillic, & chinese character sets printable somewhere in the world. Some geeks landed in the (slightly different) Gaelic alphabet, and Klingon is on UTF-8 :-/.