LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Slackware
User Name
Password
Slackware This Forum is for the discussion of Slackware Linux.

Notices


Reply
  Search this Thread
Old 06-08-2021, 11:39 AM   #1
andygoth
Member
 
Registered: Sep 2017
Distribution: Slackware
Posts: 132

Rep: Reputation: 66
Merging duplicate initrd.img and kernel files


The Slackware64 file tree contains two identical copies of initrd.img. It also has two identical kernels, albeit with different names:

Code:
$ ls -l $(find -name initrd.img) EFI/BOOT/huge.s kernels/huge.s/bzImage
-rw-r--r-- 1 root root 48023372 Jun  7 00:16 ./EFI/BOOT/initrd.img
-rw-r--r-- 1 root root 48023372 Jun  7 00:16 ./isolinux/initrd.img
-rw-r--r-- 1 root root 10744608 Jun  4 15:13 EFI/BOOT/huge.s
-rw-r--r-- 1 root root 10744608 Jun  4 15:13 kernels/huge.s/bzImage
$ sha1sum $(find -name initrd.img) EFI/BOOT/huge.s kernels/huge.s/bzImage
94e3ea10791bbd70c4a8e71f78e3effa98d55fde  ./EFI/BOOT/initrd.img
94e3ea10791bbd70c4a8e71f78e3effa98d55fde  ./isolinux/initrd.img
60a5ca08172ba756823a9c5bea4e54259e5b944b  EFI/BOOT/huge.s
60a5ca08172ba756823a9c5bea4e54259e5b944b  kernels/huge.s/bzImage
This means every time I update my local mirror and there's been a kernel/initrd update, I have to download 56 megabytes of duplicate data.

Can we eliminate the EFI copy of initrd.img and huge.s from the file tree and, in its place, modify the xorriso call in isolinux/README.TXT to use graft points? Like so:

Code:
xorriso -as mkisofs \
  [... see README.TXT for the rest ...]
  -output /tmp/slackware-dvd.iso \
  -graft-points \
  . \
  EFI/BOOT/initrd.img=isolinux/initrd.img \
  EFI/BOOT/huge.s=kernels/huge.s/bzImage
Edit: It would be good to add a README.TXT to EFI/BOOT to explain that it will be populated with initrd.img and bzImage when the image is created with the command line found in isolinux/README.TXT.

There's also usb-and-pxe-installers/usbboot.img, but it is distinct and needs to stay that way. initrd.img and bzImage, however, are low-hanging fruit when it comes to reducing bandwidth and disk storage needs for mirrors.

As an added bonus, at least with xorriso (not sure about true-blue mkisofs), this reduces the resultant disc image by 57.2 megabytes (I measured, will obviously change along with initrd.img and huge.s). I assume the grafted filenames get mapped to the same LBA. I can dive into the ISO9660 structures to confirm if need be, but I think it's self-evident. Anyway, this will be a savings for people who download, mirror, or burn the ISO images, even if they don't care about the file tree. I imagine this will encompass the majority of Slackware end users.

Mounting the disc images created with and without -graft-points and then diff'ing their file trees shows no differences. The only difference I can spot is by listing directories with "ls -li": Even though the initrd.img and kernel files have distinct inodes, they have link counts of 2 rather than 1.

At long last, I have the answer to the ancient riddle: When is a hard link not a hard link?

I've been creating and booting custom installer images created in this manner for several months now (since January) and have had no issues. I did so because I've been making and using custom initrd.img files to add vmd and custom packages and scripts I wanted/needed during installation, before the first live boot, and I found graft points to be easier and safer than commingling my custom stuff throughout the mirrored tree, which would have necessitated carefully crafting my rsync command line to protect my files.

Actually I should qualify that statement. I didn't spot the opportunity to merge kernel files until this morning, but initrd merging I've been doing all year long. To make sure it's still good, I successfully booted a fresh image created with the above command line.

Edit 2: I booted using VirtualBox using both BIOS and EFI modes. Both worked.

This discussion applies only to 64-bit Slackware since the EFI directory doesn't appear on 32-bit Slackware.

Last edited by andygoth; 06-08-2021 at 11:59 AM. Reason: suggest README.TXT, clarify testing
 
Old 06-08-2021, 12:25 PM   #2
Petri Kaukasoina
Senior Member
 
Registered: Mar 2007
Posts: 1,779

Rep: Reputation: 1459Reputation: 1459Reputation: 1459Reputation: 1459Reputation: 1459Reputation: 1459Reputation: 1459Reputation: 1459Reputation: 1459Reputation: 1459
Quote:
Originally Posted by andygoth View Post
Code:
$ ls -l $(find -name initrd.img) EFI/BOOT/huge.s kernels/huge.s/bzImage
-rw-r--r-- 1 root root 48023372 Jun  7 00:16 ./EFI/BOOT/initrd.img
-rw-r--r-- 1 root root 48023372 Jun  7 00:16 ./isolinux/initrd.img
-rw-r--r-- 1 root root 10744608 Jun  4 15:13 EFI/BOOT/huge.s
-rw-r--r-- 1 root root 10744608 Jun  4 15:13 kernels/huge.s/bzImage
$
This means every time I update my local mirror and there's been a kernel/initrd update, I have to download 56 megabytes of duplicate data.
Not, if you add -H to your rsync command:
Code:
-rw-r--r-- 2 mirror mirror 48023372 2021-06-07 05:16 ./EFI/BOOT/initrd.img
-rw-r--r-- 2 mirror mirror 48023372 2021-06-07 05:16 ./isolinux/initrd.img
-rw-r--r-- 2 mirror mirror 10744608 2021-06-04 20:13 EFI/BOOT/huge.s
-rw-r--r-- 2 mirror mirror 10744608 2021-06-04 20:13 kernels/huge.s/bzImage
(Edit: it may depend on the rsync host if the hard links are there.)

Last edited by Petri Kaukasoina; 06-08-2021 at 12:30 PM.
 
Old 06-08-2021, 02:36 PM   #3
tadgy
Member
 
Registered: May 2018
Location: UK
Distribution: Slackware (servers), Void (desktop/laptop)
Posts: 299

Rep: Reputation: 401Reputation: 401Reputation: 401Reputation: 401Reputation: 401
Quote:
Originally Posted by Petri Kaukasoina View Post
(Edit: it may depend on the rsync host if the hard links are there.)
Which they are on slackware.uk
 
Old 06-09-2021, 12:57 AM   #4
scdbackup
Member
 
Registered: Oct 2013
Posts: 158

Rep: Reputation: Disabled
Hi,

> The only difference I can spot is by listing directories with "ls -li":
> Even though the initrd.img and kernel files have distinct inodes, they
> have link counts of 2 rather than 1.

Files with identical inode on the input hard disk get their content mapped
to the same data range in the ISO. So this is much like a hard link.

With xorriso setting -compliance "new_rr" they even get the same Rock Ridge
"serial number". But the inode number in Linux is not derived from that
info. Instead the byte position of the directory record is used, divided
by 32.
(An ISO 9660 directory record holds name and pointer to data content.
So it is like a combination of dirent and inode. Such a record has at
least 34 bytes. So dividing by 32 still yields unique numbers.)

Funnily, Linux shows the number from the Rock Ridge link count field.
(Both, count and serial number, are in Rock Ridge PX entries.)

Have a nice day

Thomas
 
1 members found this post helpful.
Old 06-09-2021, 05:33 AM   #5
Petri Kaukasoina
Senior Member
 
Registered: Mar 2007
Posts: 1,779

Rep: Reputation: 1459Reputation: 1459Reputation: 1459Reputation: 1459Reputation: 1459Reputation: 1459Reputation: 1459Reputation: 1459Reputation: 1459Reputation: 1459
Quote:
Originally Posted by scdbackup View Post
Files with identical inode on the input hard disk get their content mapped
to the same data range in the ISO. So this is much like a hard link.
Yes, you are right. A Slackware tree mirrored using rsync -aH, EFI/BOOT/huge.s and kernels/huge.s/bzImage point to the same inode (with a link count of 2). Then the xorriso command line in slackware64-current/isolinux/README.TXT makes slackware-dvd.iso where those two names huge.s and bzImage point to the same zone.

But for some reason, in the original slackware64-14.2-install-dvd.iso, they point to different zones. A rebuild with the mkisofs command in slackware64-14.2/isolinux/README.TXT makes them use the same zone. Maybe P.V. built the iso using a slackware tree copied without hard links.

Last edited by Petri Kaukasoina; 06-09-2021 at 05:52 AM.
 
Old 06-09-2021, 08:02 AM   #6
scdbackup
Member
 
Registered: Oct 2013
Posts: 158

Rep: Reputation: Disabled
Hi,

> [...] in the original slackware64-14.2-install-dvd.iso, they point to
> different zones. [...] Maybe P.V. built the iso using a slackware tree
> copied without hard links.

Seems so.

Since its beginnings, libisofs underneath xorriso unites data files
which have the same pair of (dev_t, ino_t) numbers on the input disks.
There never was a way to disable this feature and afaik it was never broken.

I have an occasion where libisofs has no chance to make the connection
between files, though:
My regression tests build test ISOs from mounted original ISOs. Due to the
inode computation of Linux, each input file shows a different inode number
and thus occupies its own space in the data storage of the new ISO.
(The new ISO lives just long enough to compare its boot records and data
file content with those of the original ISO.)

---------------------------------------------------------------------------

I had a look at:
https://mirrors.slackware.com/slackw...nstall-dvd.iso

I see traces of old bugs in the ISOLINUX postprocessing program isohybrid,
option --uefi.
(Wrong GPT backup block address, pseudo-chinese GPT partition names.)

The sequence of Rock Ridge entries looks like from mkisofs or genisoimage.
(mkisofs writes RR and NM before PX. libisofs writes PX first.)

Have a nice day

Thomas
 
1 members found this post helpful.
  


Reply

Tags
efi, mirror, mkisofs



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
WHen I rebooted my laptop it is stuck at "initrd /boot/initrd.img Shadowmeph Linux - Newbie 2 03-07-2014 03:03 PM
How do I pack stage2.img into initrd.img for a PXE linux rescue? real1elmo Red Hat 12 10-14-2009 06:29 PM
How to create new initrd.gz (or initrd.img) file? kkpal Programming 2 12-10-2007 08:38 AM
Add new cciss driver module to initrd.img ,stage2.img kunalroy2002 Linux - Software 4 09-25-2007 12:09 AM
Failed to symbolic-link boot/initrd.img-2.6.18-4-486 to initrd.img Scotteh Linux - Software 8 06-01-2007 11:24 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Slackware

All times are GMT -5. The time now is 01:57 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration