Packaging proprietary software for Linux

futlib · 08-23-2010, 06:37 AM

Hi,

we are working on an independent game and are planning to support Linux, but don't know how to best package it. Would you mind sharing your experience with us?

We want only a tar.gz archive for now. No distribution specific package, no installer. We also don't want to require the user to manually install required shared libraries, so we are shipping them. We cannot distribute the files across the filesystem as the FSHS would require, so it'll all go to /opt.

Do you know of any tools that aid in packaging for Linux with support for including shared libs? We use CMake for building, but it doesn't seem to support packaging.

Our game's (let's call it foo structure currently looks like this:

foo (binary)
data

For the package, we'll also need:

foo.sh
libs

The libs directory contains all required shared libraries. foo.sh is a shell script that sets LD_LIBRARY_PATH to include the libs directory and launches the binary.

For packaging the software, we have a shell script that does the following:

Create a directory and copy foo.sh to it
Invoke the build process and make install into the new directory
Copy shared libraries from the filesystem
Package everything as tar.gz

What do you think of this approach? Are you doing it in a similar way?

There are some problems: We have to hard code all dependencies twice (once in CMake, once in the packaging script) and we have to define the version number twice (once in the source code, once in the packaging script). Any ideas?

Furthermore, we have difficulties to determine which shared libraries to ship, ldd foo shows an awfully long list. How can we figure out which libraries are supposed to be part of a typical Linux distribution and which we'll have to ship? WorldOfGoo for instance ships only very few libraries.

We thought about picking a few distributions that we want to support (probably the top 4 from distrowatch) and see with what they ship. Is there a better way?

grail · 08-23-2010, 07:27 AM

Quote:

We thought about picking a few distributions that we want to support (probably the top 4 from distrowatch) and see with what they ship. Is there a better way?

Not sure I can help on the rest, but sticking with the big 3:

rpm
apt-get
source

You probably will find it hard to wrong, plus other distros will then find a way to make this work with their packaging systems.

Sergei Steshenko · 08-23-2010, 11:18 AM

Distribute as much as possible statically linked executable.

iam_techno · 08-23-2010, 05:00 PM

Sergei,

You might also look in to makeself http://megastep.org/makeself/. It is how id software and (I think) Epic Games packaged there stuff. It's a self extracting archive that can use either gzip or bzip2 compression and it'll all you to automatically run your make/install or whatever when installing.

Quote:

Furthermore, we have difficulties to determine which shared libraries to ship, ldd foo shows an awfully long list. How can we figure out which libraries are supposed to be part of a typical Linux distribution and which we'll have to ship? WorldOfGoo for instance ships only very few libraries.

Is there a way you could post or pastebin your ldd? Things like glibc, libGl are standard on about any Linux distro.

As far as prepackaging you game, I completely agree with grail:
rpm (Red Hat/Fedora-like systems)
deb (Debian/Ubuntu-like)
source (everything + BSD + Solaris- if you'd like)

You might also consider Slackware .lzma (I think) packages too.

Hope this helps.

iamtechno

Sergei Steshenko · 08-23-2010, 05:18 PM

Quote:

Originally Posted by iam_techno

Sergei,

You might also look in to makeself http://megastep.org/makeself/. ...

I don't think the issue of static vs dynamic linking is related to the script. My point was to make the product as little dependent on the Linux distro as possible.

wje_lq · 08-23-2010, 08:24 PM

Quote:

Originally Posted by Sergei Steshenko

Distribute as much as possible statically linked executable.

wje_lq likes this.

iam_techno · 08-23-2010, 08:34 PM

Quote:

Originally Posted by Sergei Steshenko

I don't think the issue of static vs dynamic linking is related to the script. My point was to make the product as little dependent on the Linux distro as possible.

Makeself allows you to create a self extracting file like some of the installers on Windows. Whether you want to static or dynamic link your libraries is up to you. I was thinking that since you were making a proprietary product you would want to precompile as much as possible. This would just be one more distro dependent option.

Sorry for the confusion.

iamtechno

Sergei Steshenko · 08-23-2010, 08:41 PM

Quote:

Originally Posted by wje_lq

wje_lq likes this.

The latest somewhat off-topic (but still on-topic) discovery.

I am building a lot of FOSS stuff from sources with the intent to be independent of my Linux distro. The stock 'make' is v3.81. I built from sources make-3.82 and using it got hit by this:

Code:

make[4]: Entering directory `/mnt/sdb8/sergei/AFSWD_debug/build/gstreamer-0.10.30/docs/gst'
Makefile:939: *** missing separator (did you mean TAB instead of 8 spaces?).  Stop.

.

Indeed, it's 8 spaces instead of tab, though the stock 'make' somehow tolerates it. I decided to build from sources make-3.81 - the same version as the stock one. Still the same problem !

So, apparently the stock 'make' is somehow patched or whatever.

Thus, again, distribute your product in as much encapsulated form as possible, static linking being part of encapsulation.

futlib · 08-24-2010, 12:51 AM

I can't really offer the source, since it's proprietary. There will also be deb and rpm packages someday, but I don't worry about the actual package format right now, tar.gz will do. I'm focusing on my problems with shared libraries.

I've found the awesome Linux Application Checker in the meantime, and I included most libraries it complains about, but the transitive dependencies are still a lot. The resulting binary does not work on Debian lenny - probably due to the newer versions of glibc and stdc++ required by some of our libraries.

Would you recommend building such a package on an older system to make sure that it is compatible with more distributions? I don't like the thought of downgrading all of our dependencies because of that.

I tried to do static linking, but CMake doesn't really seem to support it.

iam_techno · 08-24-2010, 03:10 AM

futlib,

Quote:

Originally Posted by futlib

I can't really offer the source, since it's proprietary.

Is there anyway to post just the ldd output, censored to protect your product? Or at least the libs you have questions about?

Sergei,

I apologize. I was still half asleep when I addressed those two responses to you about makeself I meant to address them to futlib.

iamtechno

futlib · 08-24-2010, 03:36 AM

Quote:

Originally Posted by iam_techno

futlib,
Is there anyway to post just the ldd output, censored to protect your product? Or at least the libs you have questions about?

I don't think our ldd output has to be protected

Here it is (Lots of "not found" because I invoked ldd on a server):

Code:

        linux-gate.so.1 =>  (0xf57fe000)
        libboost_filesystem.so.1.40.0 => not found
        libboost_system.so.1.40.0 => not found
        libboost_program_options.so.1.40.0 => not found
        libSDL-1.2.so.0 => not found
        libpthread.so.0 => /lib/libpthread.so.0 (0xb7f5e000)
        libSDL_image-1.2.so.0 => not found
        libSDL_ttf-2.0.so.0 => not found
        libGLU.so.1 => not found
        libGL.so.1 => not found
        libSM.so.6 => not found
        libICE.so.6 => not found
        libX11.so.6 => /usr/lib/libX11.so.6 (0xb7e6e000)
        libXext.so.6 => /usr/lib/libXext.so.6 (0xb7e60000)
        libopenal.so.1 => not found
        libogg.so.0 => not found
        libvorbis.so.0 => not found
        libvorbisfile.so.3 => not found
        libvorbisenc.so.2 => not found
        libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0xb7d70000)
        libm.so.6 => /lib/libm.so.6 (0xb7d4a000)
        libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xb7d3d000)
        libc.so.6 => /lib/libc.so.6 (0xb7bff000)
        /lib/ld-linux.so.2 (0xb7f7e000)
        libxcb-xlib.so.0 => /usr/lib/libxcb-xlib.so.0 (0xb7bfd000)
        libxcb.so.1 => /usr/lib/libxcb.so.1 (0xb7be5000)
        libdl.so.2 => /lib/libdl.so.2 (0xb7be0000)
        libXau.so.6 => /usr/lib/libXau.so.6 (0xb7bdd000)
        libXdmcp.so.6 => /usr/lib/libXdmcp.so.6 (0xb7bd8000)

This was linked on Ubuntu 10.04.

My actual dependencies are:

Boost 1.40
Boost.Filesystem 1.40
Boost.Program_Options 1.40
SDL 1.2
SDL_image 1.2
SDL_ttf 2.0
OpenAL 1.12
libogg 1.1.4
libvorbis 1.2.3

The rest is transitive.

I've read the advice of building the required libraries myself in the required version and on an older distribution (like Debian lenny). At least for Boost and SDL, this would solve a couple of problems. What do you think of this approach?

futlib · 08-27-2010, 02:45 AM

I ended up doing it as follows: I installed Debian lenny into a virtual machine and compiled SDL, SDL_image, SDL_ttf and boost in the exact versions I wanted myself. This resulted in libraries with very few dependencies - just the bare necessities. I then installed these into the system using make install and compiled my application against them - CMake was able to find them in /usr/local without problems.

That's a great approach, I have only a couple of dependencies now! Linux Application Checker showed that my application is now compatible with up to three year old versions of all the major distsributions - this is certainly enough for me.