I want to make bzPrograms

lucky_dev36 · 03-02-2007, 05:01 PM

I was wondering how kernel developers did it with the kernel, it would be nice to know how to do the same for large executables, and possibly shared libraries. I am working on ls (I know, kinda basic.) to make a bzProg (or bzImage'd program.) It seems as if libc needs to be extended. (I am using zlib, sice these are userspace programs.) I am thinking C, but as I get closer, it looks like it is an ASM job,

Thanks in advance, if no criticism.

jschiwal · 03-02-2007, 05:45 PM

I'm not certain what you are doing with the ls command. You might want to look at the source for bzcat. The source for "less" or a postscript viewer may also have sections of code that decompress bzip2 compressed files on the fly.

lucky_dev36 · 03-02-2007, 08:40 PM

Quote:

Originally Posted by jschiwal

I'm not certain what you are doing with the ls command. You might want to look at the source for bzcat. The source for "less" or a postscript viewer may also have sections of code that decompress bzip2 compressed files on the fly.

I was thinking more like the program itself as the compressed file. Nevertheless when I written this, shortly after, I found UPX. It is ok to close this thread now.

P.S.: I said bzPrograms because the Program itself would be gzipped. It would have loaded it at 0x100000 (kinda smells like popcorn) and then load the main() in the decompressed program. However when I would make the program (objcopy with the --add-section option to infuse the gzipped program.) the pointers to the main program segfaulted. The project is off because UPX was found.

BTW: Does upx have better compression ratios?

gnashley · 03-03-2007, 02:13 AM

UPX is great stuff. It's what's used to compress the images from the Mars Rovers before sending back to earth. The source is not available though. But it has a little brother called exepack which is open sources so you can study the sources.
I've been improving an old program which does some of what you want. It's called pdv and creates a self extracting/executing archive or file. You can also use wrapper scripts, even for shared libraries, which will unpack and execute programs on-the-fly.

Transparent decompression is available and used many ways under linux -look at cramfs, squashfs, mkisofs using mkzftree and cloop are just a few ways. You can even (patch)configure the kernel to use compressed RAM.
Be aware that most schemes to run compressed executables will have a larger memory overhead, at least when the program starts, so be sure to use with prudence. Also because some won't run during boot-up and compressed shared libs won't always work for all programs.