[SOLVED] Reproducible builds. Chroot environment, date and config.

viel · 02-29-2024, 10:16 AM

Hi all,

Omg, i post this on the worng forum. Virtualization, sorry. Hope that are the right.

Looking for reproducible builds in the Slackware Sub-Forum I only found 3 threads.
One in 2015 other in 2016 and on 2021. But nothing relevant for mi question.

Maybe I'm little daring. Maybe I have to much free time. Maybe I hope Slackware can do it.

I am now playing with chroot to try made an environment for reproducible builds. From my understand to do a reproducible build there are 2 requirements, same date "SOURCE_DATE_EPOCH" and same environment.

I am not an expert on C, compiler, environments, ... So sorry if I'm very wrong.

From now i am trying to change the date on chroot to fix the SOURCE_DATE_EPOCH. I looking deeper to change the system date.

Code:

[root@arcadia sis]# unshare -T -- chroot /tmp/HugeSlackBuildchroot/ /bin/bash  --norc
bash-5.1# echo "boottime $((9*24*60*60)) 0" > /proc/$$/timens_offsets
bash-5.1# echo "monotonic $((2*24*60*60)) 0" > /proc/$$/timens_offsets
bash-5.1# uptime --pretty
up 4 weeks, 22 hours, 18 minutes
bash-5.1#
bash-5.1#
bash-5.1#
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
[root@arcadia v]# uptime --pretty
up 2 weeks, 5 days, 22 hours, 19 minutes
[root@arcadia v]#

Can i have different date for chroot and system?

I installed desired packages on the chroot to build a random package and checksum it. Actually I use the random ncdu to test. I need the same packages installed on the environment rigth? That are trivial to install desired packages on a chroot. Done.

I appreciate if someone can point me in the right way.

I think containers are not the way, too heavy.

I am on a rabbit hole?

Can chroot do same as a container from the point of view of making a reproducible build? Construct same environment, date, etc...

How many thinks are needed on the environment to make a reproducible build?

From env:

Code:

[root@arcadia tmp]# env
SHELL=/bin/bash
WINDOWID=1258293
XDG_CONFIG_DIRS=/etc/xdg:/etc/kde/xdg
TERM_PROGRAM_VERSION=3.2a
TMUX=/tmp/tmux-0/default,4977,0
PKG_CONFIG_PATH=/usr/local/lib64/pkgconfig:/usr/local/share/pkgconfig:/usr/lib64/pkgconfig:/usr/share/pkgconfig
GNOME_KEYRING_CONTROL=/home/xxx/.cache/keyring-3I7NI2
G_BROKEN_FILENAMES=1
HOSTNAME=arcadia.local
MINICOM=-c on
JAVA_HOME=/usr/lib64/zulu-openjdk8
SSH_AUTH_SOCK=/root/.ssh/ssh_auth_sock
ANT_HOME=/usr/share/ant
XDG_SEAT=seat0
PWD=/tmp
LOGNAME=root
XDG_SESSION_TYPE=tty
MANPATH=:/usr/lib64/zulu-openjdk11/man:/usr/lib64/zulu-openjdk8/man
UNO_PATH=/usr/lib64/libreoffice/program
XAUTHORITY=/root/.xauthasdG7U
LS_OPTIONS=-F -b -T 0 --color=auto
WINDOWPATH=1
HOME=/root
LANG=en_US.UTF-8
--squipped--

I need all the environment variables are the same? Can this be configured, automated, changed?

I am looking deeper on internet but some help are welcome.

I do not believe that I lost time if I do not get to a good end, at least I always learn something.

I wrote some garbage scripts in Github from which I am doing the tests. https://github.com/VielLosero/HugeSlackBuilds

All welcome.

Thanks in advance.

Viel.

Lockywolf · 03-01-2024, 10:59 PM

Reproducible builds are an idiotic idea. If you're rebuilding something, that's because you either want to do it machine-tailored, or you want to patch it. So there is nothing to reproduce.

If you're doing neither, don't rebuild, just use redhat.

viel · 03-02-2024, 05:16 AM

You may be right and be a waste of time from the point of view you describe, Machine-Tailoring. In that case before Red Had I went to Gentoo.

From another point of view I see it useful at the time of providing packages. Being able to reproduce the same package with the same MD5 output gives more confidence. From the user point of view, if I reproduce it, I no longer need the package of a third party, that is like that. But from the point of view of a package provider, instead of a mirror, if another provider has the same package created with the same MD5 you are sure that nothing has been added.

I do not doubt about sources or packages of anyone. I see it simply as another point of verification. Task segmentation, controls ...

You really made me think about it.

I suppose the big ones need these verification points, but I do not think it is bad if the cost/effort - feature is good. Dunno.

But the main thing is that I am trying to learn things.

For now i think i need learn containers because chroot is not the way. mmm. But i am looking deeper.

Thanks any way Lockywolf,

dchmelik · 03-02-2024, 09:07 PM

I merely use chroot alone to build/test my SlackBuilds, though I don't aim for reproducability, I'd guess if you have the exact same packages in the chroot as in your main server/workstation, they presumably(?) have the same release dates, though I don't know if installation dates would be different or count, so you might be able to do reproducible ones... unsure but it sounds interesting. What the purpose of reproducability?

viel · 03-03-2024, 02:17 AM

Quote:

Originally Posted by dchmelik

I merely use chroot alone to build/test my SlackBuilds, though I don't aim for reproducability, I'd guess if you have the exact same packages in the chroot as in your main server/workstation, they presumably(?) have the same release dates, though I don't know if installation dates would be different or count, so you might be able to do reproducible ones... unsure but it sounds interesting. What the purpose of reproducability?

From my knowledge I see reproducibility as a way for third parties to validate the construction of the package and the files that are created. And to be able to check files in a system that normally have no changes.
For example, if bash has a md5sum of xxxxxx1 and someone else can reproduce the same build, then we know that hash are ok. If for some reason that bash is changed by something, the md5 changes, if there is no reason for it, something strange will be happening.

Similar to how you can verify an exe or a dll in Windows, but with the difference that Windows builds it alone and you have to trust it and here with open sources anyone can build it but, if it is not reproducible the hash is different and also you have to trust.

I don't know if I've clarified anything. I hope so. I'm sure the link does it. https://reproducible-builds.org/

But this can be a real headache.

EDITED: I also use it to learn about build, system, environment, chroot, containers...

viel · 03-05-2024, 12:07 PM

Hi again,

I think I am so close,

From now only change the tar from man files. But don't want to edit SlackBuild script, so

Checked with https://try.diffoscope.org

Code:

a/ncdu-1.17-x86_64-1_SBo.1.tgz vs.
b/ncdu-1.17-x86_64-1_SBo.tgz
378 B
ncdu-1.17-x86_64-1_SBo.1.tgz-content vs.
ncdu-1.17-x86_64-1_SBo.tgz-content
308 B
usr/man/man1/ncdu.1.gz
264 B
filetype from file(1)
Offset 1, 1 lines modified	Offset 1, 1 lines modified
1 	gzip·compressed·data,·was·"ncdu.1",·last·modified:·Mon·Mar··4·19:08:43·2024,·max·compression,·from·Unix	1 	gzip·compressed·data,·was·"ncdu.1",·last·modified:·Mon·Mar··4·19:08:42·2024,·max·compression,·from·Unix

Slackware makepkg are prepared with SOURCE_DATE_EPOCH

Code:

unset MTIME
if [ -n "${SOURCE_DATE_EPOCH}" ]; then
  MTIME="--clamp-mtime --mtime=@${SOURCE_DATE_EPOCH}"
fi

but inside the slackBuild scripts some gzip man files don't.

I will try to modify Slackbuild gzip man files to see.

viel · 03-05-2024, 02:31 PM

Ok, now seems that I can reproduce Slackbuilds packages.

I used my slackware x86 15.0 and a virtual machine with live slackbuild x86 15.0 to check. And have the same result.

Can't upload mi HugeSlackBuild file. Renamed to .txt but seems some checks don't agree with LQ.

If someone can check or want to try. Download the raw file then make a temporal directory and run it.

https://raw.githubusercontent.com/Vi...HugeSlackBuild

This source HugeSlackBuild script will create the ncdu Slackbuild files to build the ncdu package. You can list the md5 before and check before or after if you want.

Code:

[root@arcadia ncdu]# cat ncdu.51f587bc63b1af3db43a1ce7dc683a02.07c6ea04a5b07367769de40f3432b6e8.HugeSlackBuild | grep echo
echo "0a872dbda2d79e45937e22d5c97c01d4  ./ncdu-1.17.tar.gz" | md5sum -c || exit 1
echo "b76268fc5f4a6152ed1f388e325ecfa8  ./README" | md5sum -c || exit 1
echo "67c100dc3b10f5f103dde073b89e21d7  ./slack-desc" | md5sum -c || exit 1
echo "1697e61a13fcf25a6fcabc774038203a  ./ncdu.info" | md5sum -c || exit 1
echo "64389159eb28a399df48decd70ee0018  ./ncdu.SlackBuild" | md5sum -c || exit 1

The only change I made is on ncdu.Slackbuild gzip line

Code:

# Add -n option to gzip make reproducible when GZIP_NO_TIMESTAMPS is set
gzip -n -9 $PKG/usr/man/man?/*

On the name of the HugeSlackBuild file, the fist md5 is the md5 of the HugeSlackBuild the second is the md5 from the build ncdu SlackBuild package, in that case /tmp/ncdu-1.17-x86_64-1_SBo.tgz

So check HugeSlackBuild before run it.

Code:

[root@arcadia ncdu]# md5sum ncdu.51f587bc63b1af3db43a1ce7dc683a02.07c6ea04a5b07367769de40f3432b6e8.HugeSlackBuild
51f587bc63b1af3db43a1ce7dc683a02  ncdu.51f587bc63b1af3db43a1ce7dc683a02.07c6ea04a5b07367769de40f3432b6e8.HugeSlackBuild
[root@arcadia ncdu]#

Then run it on /tmp/ncdu or some similar you know.

And check the result.

Code:

[root@arcadia ncdu]# md5sum /tmp/ncdu-1.17-x86_64-1_SBo.tgz
07c6ea04a5b07367769de40f3432b6e8  /tmp/ncdu-1.17-x86_64-1_SBo.tgz
[root@arcadia ncdu]#

From now i removed the environment file because I think this package build without.

Need check more packages to see deeper.

I would appreciate if someone could confirm that it is reproducible.

Thanks in advance.

EDITED: WARNING!! If you check on your primary system note that the date of the system will be changed. Update it after run the script or run on a virtual machine. Thanks.

Viel.

0XBF · 03-05-2024, 05:23 PM

I get my Slackware packages from several sources:

1. From the officially released packages in the Slackware 15.0 or current tree. These packages have their md5sums stored in CHECKSUMS.md5, which is then signed by Patrick using his gpg key. I can verify this at my end with gpg, so long as I trust his process and that the gpg key is not compromised. To build each package just to prove I can reproduce the same hash seems like building the entire OS from scratch. If I wanted to do that, I'd run LFS to save time.

2. Build packages from scratch using the slackbuilds.org repo. These scripts are freely readable, editable, and the source is *usually* hosted on the source provider's servers. All you can do here is verify the source's md5sum is the same as the slackbuild's documented md5sum and trust upstreams source files. Nothing to reproduce here.

3. From AlienBob's repos, which are also gpg signed by him, and he has a solid reputation for quality software built for Slackware. See point 1.

If I have to build a package from source, I'm installing the package I just built and not some other guy's version of it. At what point is "reproducible builds" useful?

I guess you could re-build all of Slackware to reproduce it's md5sums, and then you can claim that you reproduced it all and officially verify it as a third party. That only goes as far as people trust you though.

Just not seeing the point.

viel · 03-05-2024, 07:50 PM

Quote:

Originally Posted by 0XBF

I get my Slackware packages from several sources:

1. From the officially released packages in the Slackware 15.0 or current tree. These packages have their md5sums stored in CHECKSUMS.md5, which is then signed by Patrick using his gpg key. I can verify this at my end with gpg, so long as I trust his process and that the gpg key is not compromised. To build each package just to prove I can reproduce the same hash seems like building the entire OS from scratch. If I wanted to do that, I'd run LFS to save time.

2. Build packages from scratch using the slackbuilds.org repo. These scripts are freely readable, editable, and the source is *usually* hosted on the source provider's servers. All you can do here is verify the source's md5sum is the same as the slackbuild's documented md5sum and trust upstreams source files. Nothing to reproduce here.

3. From AlienBob's repos, which are also gpg signed by him, and he has a solid reputation for quality software built for Slackware. See point 1.

If I have to build a package from source, I'm installing the package I just built and not some other guy's version of it. At what point is "reproducible builds" useful?

I guess you could re-build all of Slackware to reproduce it's md5sums, and then you can claim that you reproduced it all and officially verify it as a third party. That only goes as far as people trust you though.

Just not seeing the point.

I hope that day does not arrive but if one day either due to poor intention for carelessness or by third parties, some package creator is compromised, it will be difficult to find a way to compare with another unreproducible packages that has happened.

As my English is not very good, I quote others to explain it better.

The motivation behind the Reproducible Builds project is therefore to allow verification that no vulnerabilities or backdoors have been introduced during this compilation process. By promising identical results are always generated from a given source, this allows multiple third parties to come to a consensus on a “correct” result, highlighting any deviations as suspect and worthy of scrutiny.

This ability to notice if a developer or build system has been compromised then prevents such threats or attacks occurring in the first place, as any compromise can be quickly detected. As a result, front-liners cannot be threatened/coerced into exploiting or exposing their colleagues.

I am comfortable for now with Slackware and I trust Patrick and Slackbuilds scripts, not in packages for third parties for now. If I need a package I can compile. I came to Slackware from Arch because systemd. I was a pleasant surprise to find a distribution that believes in doing things from simple ncurses helper scripts, with software that hasn't been modified from upstream development, etc.

So in that days, malware found in Arch Linux AUR Package Repository, malicious Python packages exposed in latest repository attack, etc. I only want to contribute with something that serve as something to be prepared in case these days comes. Some others distributions are working on it.

I have no server to accommodate such a number of packages nor is it my intention to replicate everything, maybe some packages for me that can be housed in Github due to size limits. If they serve someone that are welcome, if not, I have learned something doing this garbage.

So have u noticed that Slackware makepkg have SOURCE_DATE_EPOCH ?

This is not a competition is a way of collaborating to find things that have changed in a compromised system and they should not have changed.

From my comfort I do not need to replicate anything because I trust Slackware Slackbuilds and my skills, but I have always been curious and I like to help.

But maybe this is not the way.

The important thing is, if someone can replicate the package. I guess if it works in Live Slackware will work in others.

Edited: This is not for users is for package providers. I can also sign them but this signed packages don not mean that are not compromised.

Thanks anyway. I will mark this solved.

chris.willing · 03-05-2024, 08:47 PM

Quote:

Originally Posted by 0XBF

Just not seeing the point.

Maybe just scratching an itch?

We can debate forever whether any arbitrary mystery is worth the effort to untangle. All power to the OP who was intrigued enough by something to follow through and figure it out.

chris

0XBF · 03-05-2024, 10:40 PM

Thanks for taking the time to explain. I can see how that may be useful in a more complex packaging ecosystem where one or more trusted third parties are used to distribute/verify software packages.

I am fine using Slackware with its current package signing model as I mentioned earlier, but perhaps others may find uses from your project and posting these ideas here. It's certainly interesting that you can reproduce a build in such a way. I was just trying to learn why that's a thing people want in the first place.

ArchArael · 03-06-2024, 01:44 AM

@viel, regarding the AUR malware, are you referring to the malware which was found in 2018 or is it something recent that I missed?

Specifically this: https://sensorstechforum.com/arch-li...ntain-malware/

viel · 03-06-2024, 04:44 AM

Quote:

Originally Posted by ArchArael

@viel, regarding the AUR malware, are you referring to the malware which was found in 2018 or is it something recent that I missed?

Specifically this: https://sensorstechforum.com/arch-li...ntain-malware/

Yes 2018, dunno if there are some others compromised hope not.
Edited: Sorry if I scared you. Next time i will reference best.

ArchArael · 03-06-2024, 06:28 PM

No problem. I just wanted to be sure that I did not miss anything important. Thank you for replying.