Is there an official SBo HOWTO on setting up a clean chroot for working with buildscripts?

FlinchX · 02-08-2020, 05:30 AM

While reading the SBo mailing list, I randomly see replies like "I've tried the buildscript in a clean chroot and I can (or I can't) reproduce issue X".

End users just download SBo buildscripts, execute them (as root) on their systems and install the resulting packages - either by hand or by automating the process using a tool like sbopkg.

As I understand, the development/debug process of the buildscripts that end being submitted to SBo and get added to the repository, is slightly different. While it is possible to do it on a regular Slackware system, I believe the recommended way is to do it on a fully patched latest stable Slackware with no additional third party packages installed. This helps at least correctly finding out all third party dependencies that do not belong to vanilla Slackware and ensures that the package builds on a system with default settings.

Now, while there are multiple documentation sources about writing buildscripts and the official SBo repository for buildscript templates, I can't seem to find any HOWTO documenting the workflow of setting up said clean chroot "sandbox". I can't seem to find anything in the SBo FAQ either.

Is there such a HOWTO about setting up a clean chroot for developing a buildscript inside it? One that is SBo-centric and complements the SBo buildscript templates (because I believe AlienBOB does things slightly differently and his work ends in his own repository rather than SBo).

luvr · 02-08-2020, 07:23 AM

Quote:

Originally Posted by FlinchX

Is there such a HOWTO about setting up a clean chroot for developing a buildscript inside it?

Hey, now that's a great idea! I have been wondering whether it was really necessary to create a virtual machine for building SBo packages, and a chroot seems like a nice light-weight alternative.

One drawback of a chroot vs. a virtual machine seems to be the loss of the option to quickly revert to a snapshot, though. It's not clear to me, however, at this time how much of an issue that would be.

In any case, I went looking for info on setting up a Slackware chroot, and found this: Setting up a Slackware chroot.

willysr · 02-08-2020, 07:24 AM

I simply build the submitted scripts in a clean VM of latest -stable 14.2 fully patched

luvr · 02-08-2020, 07:27 AM

Quote:

Originally Posted by willysr

I simply build the submitted scripts in a clean VM of latest -stable 14.2 fully patched

And which VM manager do you run? VirtualBox?

Didier Spaier · 02-08-2020, 08:05 AM

I can't answer for Willy, but here's how I do it using Qemu:

Code:

qemu-img create -f raw slackware64-14.2.raw 14G

Then I start the VM with this script, with the slackbuilds in subdirs of $(pwd)/slackbuilds:

Code:

#!/bin/sh
qemu-system-x86_64 \
-enable-kvm \
-cpu host \
-boot menu=on \
-boot order=c \
-m 2G \
-soundhw hda \
-display gtk \
-vga std \
-fsdev local,id=vmslack,path=slackbuilds,security_model=none \
-device virtio-9p-pci,fsdev=vmslack,mount_tag=SBo \
-drive file=/data/images_iso/slackware64-14.2.raw,format=raw,media=disk \
-drive if=pflash,format=raw,readonly,file=OVMF_CODE-pure-efi.fd \
-drive if=pflash,format=raw,file=my_uefi_vars.bin \
-cdrom /data/images_iso/slackware64-14.2-install-dvd.iso

Then from the VM to mount the shared folder as /tmp/SBo:

Code:

mount -t 9p -o trans=virtio SBo /tmp/Sbo

I have a generic kernel in the VM with an initrd, as IIRC the 9p driver should be built as a module.

The lines beginning with -drive if=pflash are needed only for an EFI firmware in the VM.

Have a good day,

BW-userx · 02-08-2020, 08:45 AM

not everyone including me want to install vm then install a version of slackware just to get one slackbuild going, and having to keep it on my system until I get the urge to create another slackbuild. a guild to how to chroot into a "clean" slackware, if that is even possiable on a system that has many 3rd party already installed, I am not up on chroot'ing. but still if it is doable that seems to be a quicker way to get a test bed up and working without having a to keep a vm and working slackware vm file on the (my) the system.

as I run current (because I think slack is over due for the stable update/upgrade)

Tonus · 02-08-2020, 08:57 AM

I ended with a VM to submit my slackbuilds (very basic, needed for a software I like and wanted to get built and running with sbopkg). A chroot might be easier to clone from clean.
I have no manager, just a clean image I duplicate when needed for complex work or get cleaned with slackpkg. I have a script to launch each VM from dmenu/rofi when needed (with qemu).

hoodlum7 · 02-08-2020, 10:06 AM

The best or cleanest way to do it would be with mounting overlay filesystem and chroot before build. This would use the existing OS install, but any changes are not permanent. The file result would be an installable package.

There is a set of scripts which you can use to build a software repository it is old but still works https://idlemoor.github.io/slackrepo/. The only issue is you need to mount /tmp as another filesystem to work around restrictions on overlay filesystems.

crts · 02-08-2020, 11:07 AM

Quote:

Originally Posted by FlinchX

Now, while there are multiple documentation sources about writing buildscripts and the official SBo repository for buildscript templates, I can't seem to find any HOWTO documenting the workflow of setting up said clean chroot "sandbox". I can't seem to find anything in the SBo FAQ either.

I always use a copy of an LXC container for this. It is a bit faster than an actual virtual machine and after the build is done I can just delete/destroy the used container and start with a fresh copy for the next Slackbuild.
I am using LXC containers because I also do not know how to create a "clean" chroot environment without duplicating already installed packages. And if I am going to duplicate package installation I may just as well use LXC for isolation.

Skaendo · 02-08-2020, 11:19 AM

I use slackrepo from idlemoor. Like hoodlum7 said you need to put /tmp on it's own partition for -current, but it quickly makes and cleans it's own chroot, downloads source files, and installs dependencies into the chroot (provided that you also add them into the slackrepo slackbuild directory) for each SlackBuild. I find it very useful for working on and making new SlackBuilds.

The only thing that I still cant figure out is how to set it up to sign the packages, there was a thread about it here but I couldn't follow it.

hoodlum7 · 02-08-2020, 12:46 PM

Quote:

Originally Posted by Skaendo

I use slackrepo from idlemoor. Like hoodlum7 said you need to put /tmp on it's own partition for -current, but it quickly makes and cleans it's own chroot, downloads source files, and installs dependencies into the chroot (provided that you also add them into the slackrepo slackbuild directory) for each SlackBuild. I find it very useful for working on and making new SlackBuilds.

The only thing that I still cant figure out is how to set it up to sign the packages, there was a thread about it here but I couldn't follow it.

In your slackrepo_<XXXX>.conf file fill out the "Variables for calling gen_repos_files.sh" section.

Quote:

-------------------------------------------------------------------------------
# Variables for calling gen_repos_files.sh

# Change this to '1' if you want to enable gen_repos_files.sh
# If enabled, you *must* set correct values for all the variables below!
USE_GENREPOS='1'

# Don't change this! it is needed so gen_repos_files.sh can find your packages
REPOSROOT="$PKGREPO"

# Repository maintainer
REPOSOWNER="[Name and email address]"

# The GPG key for the repository owner can contain a different string than
# the value of $REPOSOWNER . If you leave $REPOSOWNERGPG empty, the script will
# use the value you've set for $REPOSOWNER instead to search the GPG keyfile.
REPOSOWNERGPG=""

# Under what URL is the repository accessible:
# (this is just an example, you will need to setup the http server)
DL_URL="https://[REPO URI]/current-%ARCH%"

# The title of the generated RSS feed:
RSS_TITLE="Local SlackBuild Packages"

# The logo picture used for the RSS feed:
RSS_ICON="http://www.slackware.com/~alien/graphics/blueorb.png"

# The URL linked to when clicking on the logo:
RSS_LINK="${DL_URL}/ChangeLog.txt"

# URL to the full changelog.txt:
RSS_CLURL="${DL_URL}/ChangeLog.txt"

# The descriptive text for the RSS feed:
RSS_DESCRIPTION="Local SlackBuild Packages"

# Maximum number of RSS feed entries to display:
RSS_FEEDMAX=15

# The RSS generator must use a unique feed identifier.
# Generate one for your feed by using the string returned by "uuidgen -t":
RSS_UUID="7bddb10c-1e3d-11e6-a6b4-080027396856"

# Either use gpg or gpg2:
GPGBIN="/usr/bin/gpg2"

# Optionally use gpg-agent to cache the gpg passphrase instead of letting the
# script keep it in the environment (note that if you define USE_GPGAGENT=1
# but gpg-agent is not running, you will get prompted for a passphrase every
# single time gpg runs):
USE_GPGAGENT="1"

# Generate slack-requires, slack-suggests, and slack-conflicts lines in the
# metadata files by setting FOR_SLAPTGET to "1" -- these are used by slapt-get
FOR_SLAPTGET="0"

# Follow symlinks in case the repository has symlinks like 14.0 -> 13.37
# indicating that one package works for those two Slackware releases.
# If the script does _not_ follow symlinks, then the symlinks will appear in
# the repository listing instead of the packages they point to.
FOLLOW_SYMLINKS="1"

# If the repository has separate package subdirectories then define them here.
# Separate FILELIST.TXT, MANIFEST etc.. files will be created for all of them.
REPO_SUBDIRS=""

# If you want to exclude certain directories or files from being included
# in the repository metadata, define them here (space-separated).
# Example: REPO_EXCLUDES="RCS logs .genreprc"
# .revision files were created by a previous release of slackrepo:
REPO_EXCLUDES=".revision"

Then you have to create a key for use with gen_repos_files.sh using gpg. Below is the relevant part taken from here: http://pipedot.org/article/21CM4

Quote:

As a bit of a pre-requisite, we're going to generate a GPG key so we can properly sign our packages. Normally, you simply do this using gpg --gen-key and follow the prompts, however, since I'm running this on a VM, there's a snag because it doesn't generate the needed entropy to create a random key. So I had to do this on my desktop and then move the key to the VM.

So, once you log into an actual computer and not a VM, you can run gpg --gen-key to start the setup wizard. Most likely, it's good enough to use the defaults (at least, that's what I did). Make note of your "User ID" (a combination of your real name, comment and email address). This will be used in the slackrepo config later on.

Once your key is generated, we need to move it to the VM. First, we'll package them up into an encrypted file, then move it over.

First we need to find out what your key ID is. To do this, run:

Code:gpg -K(If you're running slackrepo on a real machine and not a VM, you're now done with the GPG process)

drgibbon · 02-08-2020, 08:23 PM

Quote:

Originally Posted by hoodlum7

The best or cleanest way to do it would be with mounting overlay filesystem and chroot before build. This would use the existing OS install, but any changes are not permanent. The file result would be an installable package.

There is a set of scripts which you can use to build a software repository it is old but still works https://idlemoor.github.io/slackrepo/. The only issue is you need to mount /tmp as another filesystem to work around restrictions on overlay filesystems.

Although I know that slackrepo does it at a much more automated level, I'm working on something like that for fun (here). Early days (I don't really want to write it in shell script), but it seems to work at the moment for my testing purposes. It doesn't create a "clean" environment, but just an overlay of the present one so that any changes are isolated from the real system.

bassmadrigal · 02-08-2020, 08:47 PM

For OP's original question, I haven't done this, but I imagine the steps to create a clean chroot would stem from installing your Slackware to a specific folder and then using overlayfs to ensure the lower directory stays untouched.

Code:

mkdir /SBo-clean/{base,changes,tmp,chroot}
installpkg --root /SBo-clean/base /location/to/installmedia/slackware64/*/*.t?z
mount -o bind /dev /SBo-clean/base/dev
mount -o bind /proc /SBo-clean/base/proc
mount -o bind /sys /SBo-clean/base/sys
mount -t overlay overlay -olowerdir=/SBo-clean/base,upperdir=/SBo-clean/changes,workdir=/SBo-clean/tmp /SBo-clean/chroot
chroot /SBo-clean/chroot

I imagine the easiest method though without using a 3rd-party product would be lxc containers like crts mentioned, but I've never taken the time to figure it out.

@Skaendo, signing your packages requires getting a gpg key setup. In my HOWTO on using slackrepo, I explained how I set mine up on a VM. It takes some more effort to get a key on a VM due to lack of entropy (kinda like randomness) that exists on a normal machine. If you're having issues with that, let me know and I can try and clarify things, although it was done a bit over 3 years ago.

@hoodlum7, it's kinda funny you found that on that site. It is the thread that I originally created here and linked in the paragraph above this.

hoodlum7 · 02-08-2020, 08:58 PM

@bassmadrigal, I had seen that information before and found what I was looking for doing a quick google. I did not even think to look here.

Skaendo · 02-08-2020, 09:15 PM

Quote:

Originally Posted by drgibbon

Although I know that slackrepo does it at a much more automated level, I'm working on something like that for fun (here). Early days (I don't really want to write it in shell script), but it seems to work at the moment for my testing purposes. It doesn't create a "clean" environment, but just an overlay of the present one so that any changes are isolated from the real system.

I like it. I wanted to do something like this. Keep it up!

Quote:

Originally Posted by bassmadrigal

@Skaendo, signing your packages requires getting a gpg key setup. In my HOWTO on using slackrepo, I explained how I set mine up on a VM. It takes some more effort to get a key on a VM due to lack of entropy (kinda like randomness) that exists on a normal machine. If you're having issues with that, let me know and I can try and clarify things, although it was done a bit over 3 years ago.

I have my keys, and have for many years. I always sign my git commits. And yes I am aware of that thread, I just don't understand how (where/what) your supposed to copy the key over, and does my root account need my keys since slackrepo runs as root? Do I need to put in my password when slackrepo runs? Because if I do a full rebuild, I have about 500 slackbuilds for each 14.2 and current and it takes about 10 hours each..