kernel module on demand loading
 

Hi!

I was just wondering why my distribution loads so many kernel modules and starts several kernel threads on boot automatically when there is no devices or software that actually needs them. This seems like a waste of resources. Shouldn't it work more like when attaching a device or launching a sofware that needs a specific kernel functionality that is not compiled in the kernel, udev/kernel loads the specific driver and after the device/software is removed it automatically unloads the module/removes the thread?

BTW, does anyone know a site that describes what all those kernel threads like kstop/kintegrityd/cqueue/kmmcd/ecryptfs-kthread etc. are supposed to do?

In fact, it more or less works this way. Try connecting/disconnecting a camera etc. The thing is that a standard PC has a long list of hardware and the list of modules for that hardware is also long. There are also per-processor kernel threads, so if you have a multi-core machine, the number of threads may be also high, but they're usually idle.

A "stock" distro usually makes heavy use of kernel-modules because otherwise you have to custom-configure a kernel ... which is not what most of their users would know how to do or would want to do. (They're basically accustomed to "the Windows experience.")

So, a lot of hardware detection goes on. A lot of modules get loaded, most of whom promptly unload themselves.

You see, the distro writer has no way to know in-advance just what your system's hardware might consist of. In the Windows world, computer manufacturers take care of that chore for you, and they pre-install an "OEM" version of Windows which is properly configured for their hardware. Linux distros generally do not have that luxury.

Certainly, if you know what you are doing you can customize Linux tremendously and thus make it boot much more quickly. (My system boots in under 15 seconds, and I last rebooted it several months ago.) But distro writers want their systems to "somehow boot, on anything." An entirely different set of technical objectives.

What I find interesting is that although my system only has jfs filesystems mounted, the pdflush kernel threads are running just because they are compiled in the kernel. Linux should be "intelligent" enough to remove these threads when no ext2/ext3 volumes are mounted regardles of compiled in ext2/ext3 filesystem support.

Actually, this design-decision on the part of the driver writer might well be perfectly justifiable. Here's why...

Perhaps, just a few moments from now, you're going to plug in a USB device that's formatted with this filesystem. The driver needs to be prepared to handle it. As long as the driver is loaded (which, only in the case of a compiled-in driver, is "forever"), it must be prepared to do its job.

If, as it happens, "the driver has no job to do," then using-up a slot or two in the kernel's process-table really costs nothing at all. (Kernel processes don't even consume virtual memory.) It really does become a "who cares" decision. In this way, the choices made by the design team are, in fact, entirely reasonable.