I'm not sure of the details, but Linux 2.6 (the kernel) uses a new kernel module format with the .ko extension, rather than the standard .o module format used for previous versions of the kernel. It does less work in user-space, presumably making it more efficient.
You should have a tool called insmod.old, which automatically gets called by insmod to install old-style modules. I also suspect that you need to enable support for them in the kernel for that to work. So you might just be able to load it with
Code:
insmod svgalib_helper.o
If not, then the place to look for the particular error you're getting is /var/log/messages.
As to how to compile them, I think it's just an option to gcc; have you tried reading the gcc manpage?