Creating a Slackware chroot: installpkg --root option and doinst.sh

drumz · 09-02-2016, 10:58 PM

I install a modest amount of packages using SlackBuilds.org (125 on one machine, 133 on another currently). I run multilib on both, which can cause problems with compiling a few packages. Due to this, and also because it just "feels" right, I want to compile my SlackBuilds on a clean system.

There a few different ways to go about this, see [1] for example. I decided that creating a chroot would be best for me, using [2], [3], and [4] as my starting point. It basically boils down to doing

Code:

installpkg --root /chrootarget a/*.txz ap/*.txz d/*.txz k/*.txz l/*.txz n/*.txz x/*.txz xap/*.txz

to install all the Slackware packages into the desired chroot location. However, I noticed some inconsistent behavior with installpkg. Sometimes the doinst.sh script operates on the host system and sometimes it only modifies the new $ROOT environment. For example, the MPlayer doinst.sh is careful to call update-desktop-database and gtk-update-icon-cache relative to the new $ROOT location:

Code:

### MPlayer-1.2_20160125-x86_64-3 doinst.sh (snippet)###
# Update the desktop database:
if [ -x usr/bin/update-desktop-database ]; then
  chroot . /usr/bin/update-desktop-database usr/share/applications 1> /dev/null 2> /dev/null
fi

# Update hicolor theme cache:
if [ -e usr/share/icons/hicolor/icon-theme.cache ]; then
  if [ -x usr/bin/gtk-update-icon-cache ]; then
    chroot . /usr/bin/gtk-update-icon-cache /usr/share/icons/hicolor >/dev/null 2>&1
  fi
fi

# Update the mime database:
if [ -x usr/bin/update-mime-database ]; then
  chroot . /usr/bin/update-mime-database usr/share/mime >/dev/null 2>&1
fi

But xterm only updates the database on the host system:

Code:

### xterm-325-x86_64-1 doinst.sh ###
if [ -x /usr/bin/update-desktop-database ]; then
  /usr/bin/update-desktop-database -q usr/share/applications >/dev/null 2>&1
fi

if [ -e usr/share/icons/hicolor/icon-theme.cache ]; then
  if [ -x /usr/bin/gtk-update-icon-cache ]; then
    /usr/bin/gtk-update-icon-cache usr/share/icons/hicolor >/dev/null 2>&1
  fi
fi

It turns out that MPlayer is pretty unique - most of the packages that update some sort of cache information operate on the host system, not the new $ROOT environment. In particular, the font packages run mkfontscale and mkfontdir in the host system, not the new $ROOT. Frank Boehm [5] created a patch to installpkg that calls doinst.sh from within a chroot, but based on my experience (which I'll get to shortly), I don't think would work in all cases. In my opinion, these are bugs in the doinst.sh scripts - if possible they should all mimic MPlayer's behavior. But, I don't want to patch over a hundred doinst.sh scripts (see Additional material, below), and I don't want to patch installpkg. Is there another way?

Yes. It turns out that all but glibc-solibs of the absolutely essential packages needed for a functioning chroot can "safely" be installed using the --root option of installpkg. The few remaining needed libraries I manually copy from the host system to the new chroot environment, and then I can simply call installpkg from within the chroot. Here it is, my script that installs Slackware into a chroot without affecting the host system:

Code:

#!/bin/bash
# install-release.sh
#
# Copyright 2016 Erich Ritz
#
# This script installs Slackware64 14.2 to a chroot environment.

IFS="$(printf '\n\t')"

SERIES=(a ap d e f k kde l n t tcl x xap xfce y)
PKG_SRC=/home/erich/pkgs/slackware/slackware64-14.2/slackware64
NEW_ROOT=/root/build/14.2-release

if [ ! -d "$NEW_ROOT" ]; then
	mkdir -p "$NEW_ROOT"
fi

# These packages are safe to install using the --root option of installpkg and
# set up the initial chroot environment from which we can install all other
# packages.
BASE_PKGS=( \
	a/aaa_base-14.2-x86_64-2.txz \
	a/aaa_elflibs-14.2-x86_64-23.txz \
	a/aaa_terminfo-5.9-x86_64-1.txz \
	a/devs-2.3.1-noarch-25 \
	a/pkgtools-14.2-noarch-10.txz \
	a/etc-14.2-x86_64-7.txz \
	a/tar-1.29-x86_64-1.txz \
	a/xz-5.2.2-x86_64-1.txz \
	a/findutils-4.4.2-x86_64-1.txz \
	a/bash-4.3.046-x86_64-1.txz \
	a/coreutils-8.25-x86_64-2.txz \
	a/grep-2.25-x86_64-1.txz \
	a/sed-4.2.2-x86_64-1.txz \
	a/util-linux-2.27.1-x86_64-1.txz)

for pkg in "${BASE_PKGS[@]}" ; do
	echo " *** --- *** Installing package $pkg *** --- ***"
	installpkg --root "$NEW_ROOT" "$PKG_SRC/$pkg"
done

# Copy files needed by the chroot installpkg.  All of these will be overwritten
# when installing glibc-solibs-*.txz.
copy_file() {
	cp "$1" "$NEW_ROOT$1"
}
link_file() {
	( cd "$NEW_ROOT$1" ; ln -sf "$2" "$3" )
}
copy_file /lib64/libdl-2.23.so
link_file /lib64 libdl-2.23.so libdl.so.2
copy_file /lib64/libc-2.23.so
link_file /lib64 libc-2.23.so libc.so.6
copy_file /lib64/ld-2.23.so
link_file /lib64 ld-2.23.so ld-linux-x86-64.so.2
copy_file /lib64/libpthread-2.23.so
link_file /lib64 libpthread-2.23.so libpthread.so.0

# The only packages that touch /proc, /sys, and /dev are aaa_base and devs.
# Since they are already installed, they will be skipped when using upgradepkg
# to install all packages.  Thus, it is safe to mount /proc, /sys, and /dev in
# the chroot environment.  However, it turns out /proc is all that's needed for
# actually installing packages.
mount -t proc proc "$NEW_ROOT"/proc || exit 1
#mount -B /sys "$NEW_ROOT"/sys || exit 1
#mount -B /dev "$NEW_ROOT"/dev || exit 1

# The only package to touch /mnt is aaa_base, so use /mnt as the location of the
# Slackware tree.
mount -B "$PKG_SRC" "$NEW_ROOT"/mnt || exit 1

# Define the installpkg_chroot function.  Use upgradepkg rather than installpkg
# because upgradepkg will skip packages that are already installed - a desirable
# behavior because we don't want to reinstall aaa_base and devs packages.
installpkg_chroot() {
	chroot "$NEW_ROOT" upgradepkg --install-new "$1"
}

# Install a few more base packages, now using the chroot.
# These packages provide binaries called from various doinst.sh scripts.
installpkg_chroot /mnt/a/glibc-solibs-2.23-x86_64-1.txz
installpkg_chroot /mnt/ap/texinfo-6.1-x86_64-1.txz
installpkg_chroot /mnt/l/glib2-2.46.2-x86_64-2.txz
installpkg_chroot /mnt/l/desktop-file-utils-0.22-x86_64-1.txz
installpkg_chroot /mnt/l/shared-mime-info-1.6-x86_64-1.txz
installpkg_chroot /mnt/l/gdk-pixbuf2-2.32.3-x86_64-1.txz
installpkg_chroot /mnt/l/gtk+2-2.24.30-x86_64-1.txz
# fontconfig is used by all font packages
installpkg_chroot /mnt/l/glibc-2.23-x86_64-1.txz
installpkg_chroot /mnt/l/freetype-2.6.3-x86_64-1.txz
installpkg_chroot /mnt/l/harfbuzz-1.2.7-x86_64-1.txz
installpkg_chroot /mnt/x/libfontenc-1.1.3-x86_64-1.txz
installpkg_chroot /mnt/x/fontconfig-2.11.1-x86_64-2.txz
installpkg_chroot /mnt/x/mkfontscale-1.1.2-x86_64-2.txz
installpkg_chroot /mnt/x/mkfontdir-1.0.7-noarch-1.txz

for series in "${SERIES[@]}" ; do
	echo " *** --- *** Installing series $series *** --- ***"
	for pkgfull in $PKG_SRC/$series/*.t?z ; do
		pkg="$(basename "$pkgfull")"
		installpkg_chroot "/mnt/$series/$pkg"
	done
done

# Unmount
umount "$NEW_ROOT"/mnt
#umount "$NEW_ROOT"/dev
#umount "$NEW_ROOT"/sys
umount "$NEW_ROOT"/proc

Running the above script as ./install-release.sh 2>build_errors.txt gives only one error in build_errors.txt:

Code:

init: /dev/initctl: No such file or directory

I'm pretty sure this error is from "/sbin/init u" from the doinst.sh for sysvinit-2.88dsf-x86_64-4, which (I think) is harmless.

How did I come up with these two "special" lists of packages? BASE_PKGS are all the packages needed to be able to call installpkg. Basically bash and it's dependencies, and the various utilites (tar, xz, rev) and their dependencies. The second list (glibc-solibs, texinfo, etc.) provides the various binaries called by some of the various doinst.sh scripts.

You can see I've named this particular chroot installation 14.2-release. My plan is to, using overlayfs, layer 14.2-patched on top of this, and then add a 3rd temporary layer for all my SBo builds. That part I haven't gotten to yet.

[1] http://www.linuxquestions.org/questi...ent-4175585169
[2] http://docs.slackware.com/howtos:gen...ackware_chroot
[3] http://www.linuxquestions.org/questi...8/#post4093831
[4] http://www.linuxquestions.org/questi...9/#post5579695
[5] http://alt.os.linux.slackware.narkiv...nstallpkg-12-1

--- Additional material ---
Before I realized I could simply get the doinst.sh scripts from /var/log/scripts I created this script to extract them:

Code:

#!/bin/sh
# extract_doinst.sh
#
# Copyright 2016 Erich Ritz
#
# Portions of this script are copied from /sbin/installpkg provided by
# Slackware 14.2.
#
# This script extracts the doinst.sh file from a list of Slackware packages
# given as arguments to this script into the current directory.

# Return a package name that has been stripped of the dirname portion
# and any of the valid extensions (only):
pkgbase() {
        # basename + strip extensions .tbz, .tgz, .tlz and .txz
        echo "$1" | sed 's?.*/??;s/\.t[bglx]z$//'
}

#  Ext   Format  Tar flag
# .tbz - bzip2 - -j
# .tlz - lzma  - --lzma
# .txz - xz    - -J
# .tgz - gzip  - -z

# Main loop:
for package in "$@" ; do
        if [ ! -f "$package" ]; then
                echo "Cannot process $package:  file not found"
                continue;
        fi

        # "shortname" isn't really THAT short...
        # it's just the full name without ".t{gz,bz,lz,xz}"
        shortname="$(pkgbase "$package")"

        # Determine extension
        packageext="$( echo "$package" | rev | cut -f 1 -d . | rev)"

        # Set propper tar extraction flag
        case $packageext in
                'tgz' )
                        TAR_FLAG=-z
                        ;;
                'tbz' )
                        TAR_FLAG=-j
                        ;;
                'tlz' )
                        TAR_FLAG=--lzma
                        ;;
                'txz' )
                        TAR_FLAG=-J
                        ;;
        esac

        tar -x ${TAR_FLAG} -f "$package" install/doinst.sh 1> /dev/null 2> /dev/null
        if [ $? -eq 0 ] ; then
                mv install/doinst.sh "./${shortname}-doinst.sh"
                rmdir install
        else
                echo "Package $shortname does not have a doinst.sh"
        fi
done

And then this script will filter down to "interesting" lines in each doinst.sh which potentially modifies the host system instead of the new $ROOT environment:

Code:

#!/bin/sh
# analyze_doinst.sh
grep -v "^#" *-doinst.sh  | grep " /" > root-mods.txt
grep -v "ln -sf" root-mods.txt > root-mods2.txt

This leaves 511 lines from over hundred packages to analyze. But if you do, you'll quickly see the offending lines fall into just a few different categories (except for glibc - it's a beast). From this analysis I also found that just 1 doinst.sh actually had the executable bit set in the tarball package, netpipes-4.2-x86_64-1.

55020 · 09-03-2016, 01:30 AM

Quote:

Originally Posted by drumz

But xterm only updates the database on the host system:

Code:

### xterm-325-x86_64-1 doinst.sh ###
if [ -x /usr/bin/update-desktop-database ]; then
  /usr/bin/update-desktop-database -q usr/share/applications >/dev/null 2>&1
fi

if [ -e usr/share/icons/hicolor/icon-theme.cache ]; then
  if [ -x /usr/bin/gtk-update-icon-cache ]; then
    /usr/bin/gtk-update-icon-cache usr/share/icons/hicolor >/dev/null 2>&1
  fi
fi

Sorry, but you're wrong. Look where it says 'usr/share/applications' and 'usr/share/icons/hicolor'. You'll see there is no '/'.

This is highly important and highly significant, installpkg and upgradepkg have already done a cd to the specified root directory, so it is using the host system's program (/usr/bin/update-desktop-database) to update the database in the alternative root (usr/share/applications).

This means that you can (usually) update an x86_64 root from an i586 host, and even use an Intel installation to upgrade packages on a mounted Slackware ARM filesystem. You should not confuse --root and chroot. They are not the same thing, you've forgotten about the much more important use case of updating another mounted filesystem.

Basically, it all works better how it is right now. Sorry.

drumz · 09-03-2016, 11:04 AM

Quote:

Originally Posted by 55020

Sorry, but you're wrong. Look where it says 'usr/share/applications' and 'usr/share/icons/hicolor'. You'll see there is no '/'.

This is highly important and highly significant, installpkg and upgradepkg have already done a cd to the specified root directory, so it is using the host system's program (/usr/bin/update-desktop-database) to update the database in the alternative root (usr/share/applications).

Basically, it all works better how it is right now. Sorry.

Sorry, you're right; xterm was a terrible example. Here's wqy-zenhei-font-ttf:

Code:

# There's no need to chroot and do this during initial
# install, since there is a post-install script that
# does the same thing, saving time.
# Update X font indexes and the font cache:
if [ -x /usr/bin/mkfontdir ]; then
  /usr/bin/mkfontscale /usr/share/fonts/TTF
  /usr/bin/mkfontdir /usr/share/fonts/TTF
fi
if [ -x /usr/bin/fc-cache ]; then
  /usr/bin/fc-cache /usr/share/fonts/TTF
fi
( cd etc/fonts/conf.d ; rm -rf 44-wqy-zenhei.conf )
( cd etc/fonts/conf.d ; ln -sf ../conf.avail/44-wqy-zenhei.conf 44-wqy-zenhei.conf )

There is a note embedded in this script (which I haven't noticed before) here that during a "normal" installation it doesn't matter that the "wrong" font directory is used for mkfontscale and mkfontdir because the post-install script will take care of it. That's all well and good for a normal installation, but not necessarily for a chroot installation.

And here is blinken for another example:

Code:

if [ -x /usr/bin/update-desktop-database ]; then
  /usr/bin/update-desktop-database /usr/share/applications >/dev/null 2>&1
fi

( cd usr/doc/HTML/en/blinken ; rm -rf common )
( cd usr/doc/HTML/en/blinken ; ln -sf /usr/share/doc/HTML/en/common common )

(Edit: "necessarily" -> "not necessarily"

drumz · 09-03-2016, 11:09 AM

Clearly pkgtools excels at installing new systems and keeping systems up-to-date. It has decades of experience. But my point is are there any gotchas when installing into a chroot? I believe yes there are, and installing from within the chroot avoids potential problems.

55020 · 09-03-2016, 12:13 PM

Quote:

Originally Posted by drumz

There is a note embedded in this script (which I haven't noticed before) here that during a "normal" installation it doesn't matter that the "wrong" font directory is used for mkfontscale and mkfontdir because the post-install script will take care of it. That's all well and good for a normal installation, but not necessarily for a chroot installation.

From what I've read, mkfontscale, mkfontdir and fc-cache only matter for really old applications (gtk+1 etc) [see e.g. https://wiki.archlinux.org/index.php...l_installation ff]. In any case, you're not going to be running X applications in that root.

Quote:

Originally Posted by drumz

And here is blinken for another example:

Code:

if [ -x /usr/bin/update-desktop-database ]; then
  /usr/bin/update-desktop-database /usr/share/applications >/dev/null 2>&1
fi

That one is a bug; you might want to report it to Patrick. But you're not going to be running a desktop in that root, so it makes no significant difference.

USUARIONUEVO · 09-03-2016, 03:26 PM

i talk arround this in other post ..but no lucky.

installpkg using root parameter not working ever.

glibc , install this in separate dir and check how create files out of folders.

see my thread ...but no solution.

http://www.linuxquestions.org/questi...on-4175587290/

atelszewski · 09-03-2016, 04:19 PM

Hi,

You might try to use installer's initrd to do the whole thing.
This way, you would have an environment as if you were installing from DVD.

Have a look at the necessary steps I described here for remote server. Instead of installing on remote server as the text explains, you'll be installing on local machine. Also you don't have to partition the drive and instead of setup, you can probably use installpkg only.

What I mean, is that by using installer's initrd and chroot'ing into it, you should have the "proper" installation environment and all the necessary tools.

--
Best regards,
Andrzej Telszewski

drumz · 09-03-2016, 05:20 PM

Quote:

Originally Posted by USUARIONUEVO

installpkg using root parameter not working ever.

glibc , install this in separate dir and check how create files out of folders.

see my thread ...but no solution.

http://www.linuxquestions.org/questi...on-4175587290/

USUARIONUEVO, I think "installpkg using root parameter not working ever" is too strongly worded. installpkg basically does two things: extracts a package's contents into $ROOT, and then runs doinst.sh. (It also does housekeeping tasks like writing to var/log/packages and var/log/scripts.) Extracting the package contents into $ROOT is done correctly. It is the responsibility of the doinst.sh to do things "correctly" with respect to $ROOT. For me it's hard to understand your English, but I think one of the points you bring up are the behavior of symlinks. For example, let's look at the usr/share/zoneinfo/localtime symlink created by glibc's doinst.sh and what it looks like in the installed system:

Code:

( cd usr/share/zoneinfo ; rm -rf localtime )
( cd usr/share/zoneinfo ; ln -sf /etc/localtime localtime )

As 55020 pointed out, the doinst.sh script is run from $ROOT. So that snippet of code changes to $ROOT/usr/share/zoneinfo and deletes localtime and then creates a symlink pointing to /etc/localtime. Let's look at my current chroot Slackware install to see the result:

Code:

# ls -l /root/build/14.2-release/usr/share/zoneinfo/localtime 
lrwxrwxrwx 1 root root 14 Sep  2 15:13 /root/build/14.2-release/usr/share/zoneinfo/localtime -> /etc/localtime

I think what you (USUARIONUEVO) are saying is that usr/share/zoneinfo/localtime is pointing potentially outside "$ROOT" because it is linked to /etc/localtime. But I don't think this is a problem; I think it is correct behavior. When you change into the chroot, the symlink still points to /etc/localtime, but because you are inside the chroot environment, that resolves to $ROOT/etc/localtime (with respect to the host system's view), because / is viewed from within the context of the chroot's environment. If your goal is simply to examine package contents, I agree with ponce that you should use explodepkg and then examine doinst.sh yourself to see what additional tasks are done. Usually it's just adding symlinks.

But, you do correctly point out that a doinst.sh can potentially modify the host system - nothing in pkgtools prevents that from happening. It is up to each individual doinst.sh to only modify the system relative to $ROOT. That brings up a good question: what are the goals and non-goals of pkgtools? Perhaps using pkgtools to manage a chroot Slackware install is a non-goal. To me that's perfectly acceptable, I've just never seen it stated anywhere.

However, my goal is to create a full Slackware install for a chroot, using pkgtools. Based on what I've read elsewhere (as documented in my first post) I thought it would be as simple as "installpkg --root $MY_NEW_ROOT */*.t?z". But I now have a new conclusion, as documented in my install-release.sh. Install a minimum number of packages using installpkg --root, manually examining the doinst.sh scripts to make sure they don't touch the host system, and then install everything else while inside the chroot. In this way I don't have to worry about what the doinst.sh scripts do, because they are called from within a chroot.

Andrzej, that is a good suggestion! I will have to look into it. That should also guarantee that the host system isn't inadvertently modified by a doinst.sh script.

USUARIONUEVO · 09-03-2016, 06:38 PM

sorry for my bad english.

basically i say same as you , if someone, use --root parameter , doinst.sh can do something strange in some cases.

when i put --root=MYFOLDER y spect ,doinst.sh go to work inside $MYFOLDER , same as when run in to / but sometimes "post-install",actions work out or bad , in $MYFOLDER.

if you test to install glibc , in a custom folder. ... you can see , symlinks OUT OF $MYFOLDER/etc ..point to good dir, but symlinks , out of spected dir.

spected $MYFOLDER/etc/symlinks

reality $MYFOLDER/symlinks

SEE , symlinks not in /etc ... :=)

gnashley · 09-04-2016, 02:27 AM

Yes, the glibc doinst.sh does not play nicely in this case. It thinks that you are upgrading your running system. It tries to determine whether it is being run from the install initrd or a running system. As written, the test thinks that you are upgrading because /sbin/ldconfig exists. On my system, I have reworked this test so that it does the right thing when installing using ROOT -it creates the links manually instead of running ldconfig.