Current: mkisofs/isohybrid versus p7zip and engrampa

burdi01 · 05-28-2016, 08:57 AM

It looks like that ISOs mastered with mkisofs/isohybrid from Current make p7zip and engrampa behave abnormally.
Please see the https://github.com/mate-desktop/engrampa/issues/128 discussion.

Didier Spaier · 05-28-2016, 12:29 PM

FYI I have suggested to use xorriso instead of mkisofs/isohybrid to build the Slackware Live ISOs but Eric does not want to use a software not shipped in Slackware for that. I then suggested by email to another Slackware contributor to ship xorriso in Slackware but received no answer.

scdbackup · 05-29-2016, 02:34 AM

Hi,

i downloaded
http://slackware.uk/people/alien-sla...te-current.iso
for inspection.

The isohybrid program which added partition tables is still the buggy
version which publishes parts of its virtual RAM as pseudo-chinese
partition names.
This and other bugs were fixed 2014-06-24. See the changesets of this
date in
http://git.zytor.com/syslinux/syslin...ls/isohybrid.c

It might be worth to try what happens if one applies a contemporary
isohybrid program to the problematic ISO and re-tries with 7z.

The partition tables in isohybrid ISOs are somewhat irregular by having
nested partitions and GPT with non-protective MBR. But this is a feature
which is present in xorriso made isohybrids, too.

I riddle about the alleged size 8,796,092,989,440 which is beyond any
limit for MBR partitions. The maximum size of an ISO filesystem is
8,796,093,022,208. So i guess the reported number stems from a default
value in 7z for ISO image sizes.

Have a nice day

Thomas

scdbackup · 05-29-2016, 04:30 AM

Hi,

after staring a while at
http://sources.debian.net/src/p7zip/...soIn.cpp/#L604
which obviously checks the block addresses of all data files, i ran

Code:

   xorriso -indev slackware64-live-mate-current.iso \
           -find / -sort_lba -exec report_lba -- \
     2>/dev/null | head -3

and got

Code:

   Report layout: xt , Startlba ,   Blocks , Filesize , ISO image path
   File data lba:  0 ,      -16 ,        0 ,        0 , '/EFI/BOOT/SLACKWARELIVE'
   File data lba:  0 ,       62 ,      720 ,  1474560 , '/boot/syslinux/efiboot.img'

Yikes. An empty data file with start LBA -16. What a heavy test for
stability. (I will let xorriso gnaw on this ISO and expect interesting
failures.)

This matches well the alleged size number reported by 7z:
Max ISO size 8,796,093,022,208 - alleged size 8,796,092,989,440 = 32768
which is 16 * 2048 and thus what i'd expect from a unsigned 32 bit rollover
caused by block address -16.

This block address was most probably not written by program isohybrid but
rather by the ISO producing program (mkisofs or genisoimage ?).
I could not reproduce this by genisoimage-1.1.11 of Debian 8

Code:

  genisoimage -R -o test.iso empty_file

So the riddle remains what weird program produced that ISO.

Have a nice day

Thomas

Didier Spaier · 05-29-2016, 05:13 AM

Hi Thomas

Quote:

Originally Posted by scdbackup

This block address was most probably not written by program isohybrid but
rather by the ISO producing program (mkisofs or genisoimage ?).
I could not reproduce this by genisoimage-1.1.11 of Debian 8

Code:

  genisoimage -R -o test.iso empty_file

So the riddle remains what weird program produced that ISO.

It's mkisofs:

Code:

# Determine whether we add UEFI boot capabilities to the ISO:
if [ -f boot/syslinux/efiboot.img ]; then
  UEFI_OPTS="-eltorito-alt-boot -no-emul-boot -eltorito-platform 0xEF -eltorito-boot boot/syslinux/efiboot.img"
else
  UEFI_OPTS=""
fi

# Time to determine the output filename, now that we know all the variables
# and ensured that the OUTPUT directory exists:
OUTFILE=${OUTFILE:-"${OUTPUT}/${DISTRO}${DIRSUFFIX}-live${ISOTAG}-${SL_VERSION}.iso"}
mkisofs -o "${OUTFILE}" \
  -R -J \
  -hide-rr-moved \
  -v -d -N \
  -no-emul-boot -boot-load-size ${BOOTLOADSIZE} -boot-info-table \
  -sort boot/syslinux/iso.sort \
  -b boot/syslinux/isolinux.bin \
  -c boot/syslinux/isolinux.boot \
  ${UEFI_OPTS} \
  -preparer "$(echo $LIVEDE |sed 's/BASE//') Live built by ${BUILDER}" \
  -publisher "The Slackware Linux Project - http://www.slackware.com/" \
  -A "${DISTRO^}-${SL_VERSION} for ${SL_ARCH} ($(echo $LIVEDE |sed 's/BASE//') Live $VERSION)" \
  -V "${MEDIALABEL}" \
  -x ./$(basename ${LIVE_WORK}) \
  -x ./${LIVEMAIN}/bootinst \
  -x boot/syslinux/testing \
  .

assuming there was no commit since you downloaded the ISO.
The file is there: http://bear.alienbase.nl/cgit/livesl...ckware_live.sh

Off topic: I made ISOs bootable on UEFI firmware using elilo with a menu instead of grub. I will soon send you a link so you can add ore or two to your collection

Have a nice day.

Alien Bob · 05-29-2016, 05:45 AM

Quote:

Originally Posted by scdbackup

The isohybrid program which added partition tables is still the buggy
version which publishes parts of its virtual RAM as pseudo-chinese
partition names.
This and other bugs were fixed 2014-06-24. See the changesets of this
date in
http://git.zytor.com/syslinux/syslin...ls/isohybrid.c

Those bugfixes were never released as part of a syslinux 4.x unfortunately.

Didier Spaier · 05-29-2016, 06:32 AM

Quote:

Originally Posted by Alien Bob

Those bugfixes were never released as part of a syslinux 4.x unfortunately.

Maybe it's about time to release Syslinux 4.0.8, nearly 3 years after 4.0.7.

Thomas, would you mind requesting that on the mailing list? You can certainly state a more convincing rationale for a release than me.

As a workaround, maybe Slackware could just ship just a newer isohybrid. I know that generally Ady warns against using components not belonging to the same release, but maybe this wouldn't hurt in that case. Thomas, what is your advice on that, knowing that Slackware 14.2 will ship Syslinux 4.0.7?

scdbackup · 05-29-2016, 06:45 AM

Hi,

Didier Spaier wrote:

Quote:

it's mkisofs

Code:

... -eltorito-platform 0xEF ...

Quite a young one, as it seems.

Ahum. It seems to be no young behavior. I found an old mkisofs-2.01.01a64
which already reproduces the problem. The empty file ends up with
block address -16 or 4294967280 or 0xfffffff0.

In cdrtools-2.01.01a64/mkisofs/mkisofs.h i see

Code:

  #define        NULL_INO_MAX    ((UInt32_t)0xFFFFFFF0)

So i would advise to check whether younger mkisofs versions use a
better block address for empty data files or non-data files like
symbolic links or device files.

Further i would advise to file a bug to 7z which asks to omit files
of length 0 from the assessment of highest used data block address.
I believe to see this assessment in
http://sources.debian.net/src/p7zip/...soIn.cpp/#L604
(using the Debian repository because it offers code search).

---------------------------------------------------------------------

I wrote:

Quote:

The isohybrid program [...] is still the buggy version

Alien Bob wrote:

Quote:

Those bugfixes were never released as part of a syslinux 4.x unfortunately.

Argh. I nagged against the bugs a few years before hpa was annoyed enough.
Now the fixes are sitting in git.

Well, for this topic it seems not to be the decisive difference between
xorriso's isohybrid and mkisofs+isohybrid's isohybrid.
7z rather has the problem that empty files point out of the image data
block range. This does not hurt typical use of mounted ISOs because
any reasonable driver will not address that block if there are no
bytes to read from it. (Actually i know no way to do this on SCSI level.)

isohybrid is not to blame. mkisofs and 7z both lean out of the train
too far and collide where nobody is supposed to travel.

Have a nice day

Thomas

scdbackup · 05-29-2016, 07:33 AM

Hi,

Didier Spaier wrote:

Quote:

Maybe it's about time to release Syslinux 4.0.8, nearly 3 years after 4.0.7.

Thomas, would you mind requesting that on the mailing list?
You can certainly state a more convincing rationale for a release than me.

Actually you may well have a better standing than me.

After all you know how use ISOLINUX whereas i only know how to pack
it up in an ISO. My nagging towards isohybrid.c finally got approved
by hpa. This fact should sum up and outweight any technical explanation
of mine.

We could both show up on behalf of this matter. But i am sure that others
have more pressing reasons to wait for a SYSLINUX release. A sincere
attempt to get a new (legacy ?) release should probably give more
motivation than only isohybrid.
(I am not aware of the policy regarding version 4, 5, 6. Last release
of version 6 seems nearly 2 years ago.)

Quote:

I know that generally Ady warns against using components not belonging to the same release, but maybe this wouldn't hurt in that case.

There is the theoretical prescription by hpa to use the isohybrid program
that came with the SYSLINUX package from where the isolinux.bin boot image
file stems. This is because isohybrid programs contain MBR x86 machine
code copies which must prepare the jump from BIOS to isolinux.bin.

The new version in git knows an option --mbr by which one can select
one of the MBR files as mentioned in
http://www.syslinux.org/wiki/index.p...#MBR_selection
instead of one of the program-internal MBR blobs which get selected
by the corresponding options of isohybrid.
So one can be sure to have matching MBR and isolinux.bin.

For repacking existing ISOs, i cut out the first 446 bytes from the
original and use this file as input for option -isohybrid-mbr.

Have a nice day

Thomas

Alien Bob · 05-29-2016, 10:03 AM

To help people that experience issues with the mkisofs-generated ISO images, I am adding xorriso support to my "make_slackware_live.sh" script: install xorriso and then add the switch "-X" to the make_slackware_live.sh command to make it invoke xorriso.
I tested it locally and it works - at least, the ISO boots in a VM. I will probably upload a set of ISO images generated with xorriso to have people test it. What I still have to test & compare: whether the hybrid ISO boots a computer when dd-ed to a USB stick.

The default action for the script will still be to use mkisofs/isohybrid, even when xorriso is installed.

burdi01 · 05-29-2016, 11:00 AM

The heavy testing I have been doing in the meantime would indicate that mkisofs and *not* isohybrid is the determining factor.

Using the attached mkgriso script (to be placed in the root of the ISO) I tested with mkisofs 3.01a08, 3.01a17, 3.01 and 3.02a06 and found that with all versions p7zip and engrampa show the problems.

I then "stole" genisoimage (1.1.11) from Xubuntu 14.03.3, ran the mkgriso script again and lo and behold: p7zip and engrampa no longer complain.

Note that with all tests /usr/bin/isohybrid was/is the "stock" one from Current.

PS: In an earlier discussion the attached mkxoriso script (also to be placed in the root of the ISO) was reported to be OK.

scdbackup · 05-29-2016, 12:23 PM

Hi,

thank you for flying xorriso.

Regarding mkgriso i have to remark that "original" genisoimage-1.1.11
does not know option -e for announcing EFI El Torito boot images.
I understand it was Fedora who introduced it in its version.

One may use xorriso -as mkisofs with the same options as the enhanced
genisoimage and leave it to program isohybrid to make the ISO bootable
from USB stick.
So mkgriso could make use of all three: mkisofs, genisoimage, xorriso.

Have a nice day

Thomas

Alien Bob · 05-29-2016, 01:54 PM

I have added xorriso support to liveslak in this git commit: http://bear.alienbase.nl/cgit/livesl...3a9591026663a9