Package managers and how to use them

A surprising number of Linux newbies do not seem to understand how to use a package manager despite the fact that they probably have smartphones and are therefore likely to be quite familiar with the concept of an app store.

Linux was using app stores long before Steve Jobs created the iPhone. However in the Linux world they are called repositories (repo for short).

Windows users tend to acquire bad habits where software installation is concerned. They browse about the Web using IE or Firefox and download advertised software as zipped archives, which they then unzip and install by running whatever installation program the package provides. Typically this will be called something like install.exe or setup.exe. It's a binary program so you can't look inside it to see what it does. If you are lucky, it will simply copy the other files to their allotted places and make a few registry edits. If you are less lucky, it will infect your system with worms or viruses.

You are less likely to get infected if you download your software only from reputable sites, but in the world of proprietary software, reputable sites tend to demand payment for their programs. Browse around a bit though, and you can usually find cost-free versions of the same packages. They are pirated of course, and the reason they come free is because they are bait, like the Trojan Horse that has given its name to a whole class of malware.

When these people start using Linux, they often continue to download packages which they find on various sites. Mostly they know enough to choose the right binary format (for example a .deb file for Debian-based distros) but they have no idea how badly this can mess up their systems.

A Linux system is tightly integrated. In Windows every program tends to come with its own built-in libraries (.dll files), simply because there are legal prohibitions on using libraries that were created by other companies. In Linux, there is a single underlying layer of libraries which all the programs use. But each program requires its run-time libraries to be the ones it was built against. Minor release changes can be accommodated; major ones cannot. Therefore all the main program packages must be in compatible versions that can seamlessly share this single layer of libraries. Ensuring this is an important part of a distro developer's job.

Every time you download a binary package by hand and install it you risk screwing up your system. If it is (or contains) a library, it may replace a different version of that library which you already have. It may or may not prove compatible with all your existing applications. Or there may be no existing application on your system that uses it but you may later wish to install such an application. Since your package manager has no way of knowing that you have installed that library, it will try to install the system version, which may then screw up the application you downloaded.

If the package is an application, it may require libraries that you do not have. Or it may come with its own built-in libraries, which are not of good quality. Firefox and FF clones such as Pale Moon are actually designed to be downloaded and installed by hand. They come with the libraries they need and install them in a non-standard location where they will not clash with the system libraries. But because these libraries are not actually part of the project but merely copied over from their own sites, the program developers often fail to keep them up to date, and that has security implications. It is much better therefore to install such programs via the package manager so that they can use the system libraries and take advantage of their automatic updating.

You also need to remember that when you install things yourself, you are going behind your package manager's back (as it were). The result is that the package manager's database of installed software is no longer accurate, and this could have repercussions down the line. For example, if a package is due for an upgrade, it will not be upgraded if the package manager doesn't know that you have it installed.

How do package managers actually work? Well, typically there are two layers. The simpler (and often older) one is a program that understands the packaging format and can therefore unpack and install the software correctly. It knows which files to overwrite in the case of an update (and which must not be overwritten), and also how to remove the software completely from the system if required. On most systems, the low-level package manager also handles dependency checking. It will not install packages whose dependencies are not met.

The higher-level package manager (sometimes called an update manager) is responsible for fetching the software and handling updates automatically. It has a configuration file (somewhere in the /etc directory tree) with the addresses of the repositories, and it can synchronise the local software database with the remote one, so that it knows which installed packages have pending updates. The new versions will then be downloaded automatically and the lower-level manager will be called upon to install them. Any additional packages required as dependencies will be downloaded too. You can also use your high-level package management to search the repositories for particular software packages by name or by subject.

Each family of Linux distributions has its own package management system. For example all Debian derivatives (Ubuntu, Mint, AntiX, ElementaryOS, Pinguy, Bodhi and many others) use dpkg as a low-level manager with a choice of three high-level ones: apt-get (command line), aptitude (menu-based) or synaptic (graphical). Red Hat distros such as Fedora and Centos use rpm (low level)and yum (high level). Arch has Pacman, Gentoo has Portage, and so on.

It is important to note that just because two distros use the same package format, it does not mean that their software packages can be mixed and matched. For example the Debian Stable and Unstable distros and the various releases of Ubuntu all use different repositories. The same packages occur in most of them but they will almost certainly be different versions. If you trick your package manager into using non-standard repositories, you can seriously bork your system.

If you want to use a program that is not already on your system, your first port of call is your package manager. But what happens if the package is not available? This is unlikely to happen in distros like Debian and Ubuntu, which pride themselves on providing a wide range of software. It might well happen in smaller distros.

There are a few special programs like Firefox and LibreOffice that can be installed as self-contained systems, with the caveats noted above. There are also recently developed systems like snaps and appimages which do something similar for a wider range of programs. But for most software that is not an option. So, if you absolutely must have that program and none of the available alternatives will do, where do you go? The answer is that you must then download the source code and build the binary package locally. This is a lot less difficult than it sounds because Linux has a built-in automated system (the GNU build system) which practically all Linux software uses. Once you have the necessary tools installed, compiling and installing is usually as simple as:
The built-in configuration script will check that you have the necessary libraries on your system (if not, your package manager can probably install them). As a result, the installed package will be perfectly integrated with the rest of your software. The one weakness of this method is that if you install by hand, your package manager will not know about the package. Fortunately many distros have tools that will let you complete a build from source by creating a binary package in the distro's own package format. This can then be installed in the usual way.