usbimg2disk.sh in slack64 current

WiseDraco · 11-08-2015, 05:47 AM

Hello!
Today i download slackware64 current to my usb flash for install on brothers notebook:

lftp -c 'open http://mirrors.slackware.com/slackwa...kware-current/ ; mirror -c '

then i copy folder
/usb-and-pxe-installers from that flash to my desktop pc, where that flash is inserted ( slackware64 14.1 )

then i umount usb flash drive, and try make them bootable:

/mypath/usb-and-pxe-installers/usbimg2disk.sh -i usbboot.img -o /dev/sdd

then i get that strange error:

bash-4.2# sh usbimg2disk.sh -i usbboot.img -o /dev/sdd

# We are going to use this device - '/dev/sdd':
# Vendor : UFD 2.0
# Model : Silicon-Power8G
# Size : 7681 MB
#
# FDISK OUTPUT:
#
# Disk /dev/sdd: 8054 MB, 8054112256 bytes
# 1 heads, 62 sectors/track, 253720 cylinders, total 15730688 sectors
# Units = sectors of 1 * 512 = 512 bytes
# Sector size (logical/physical): 512 bytes / 512 bytes
# I/O size (minimum/optimal): 512 bytes / 512 bytes
# Disk identifier: 0xc3072e18
#
# Device Boot Start End Blocks Id System
# /dev/sdd1 * 24 15730687 7865332 c W95 FAT32 (LBA)

*** ***
*** If this is the wrong drive, then press CONTROL-C now! ***
*** ***
Or press ENTER to continue:
usbimg2disk.sh: line 363: UFD: command not found
*** usbimg2disk.sh FAILED at line 363 ***
--- Cleaning up the staging area...
bash-4.2# sh usbimg2disk.sh -i usbboot.img -o /dev/sdd1
*** I fail to find a 'vfat' partition: '/dev/sdd11' !
*** If you want to format the USB thumb drive, add the '-f' parameter

see at line 363 and - there is something about a labelling.
ok, try to give label in command:

bash-4.2# sh usbimg2disk.sh -L SLCK64_cur -i usbboot.img -o /dev/sdd

# We are going to use this device - '/dev/sdd':
# Vendor : UFD 2.0
# Model : Silicon-Power8G
# Size : 7681 MB
#
# FDISK OUTPUT:
#
# Disk /dev/sdd: 8054 MB, 8054112256 bytes
# 1 heads, 62 sectors/track, 253720 cylinders, total 15730688 sectors
# Units = sectors of 1 * 512 = 512 bytes
# Sector size (logical/physical): 512 bytes / 512 bytes
# I/O size (minimum/optimal): 512 bytes / 512 bytes
# Disk identifier: 0xc3072e18
#
# Device Boot Start End Blocks Id System
# /dev/sdd1 * 24 15730687 7865332 c W95 FAT32 (LBA)

*** ***
*** If this is the wrong drive, then press CONTROL-C now! ***
*** ***
Or press ENTER to continue:
--- Setting FAT partition label to 'SLCK64_cur'
--- Available free space on the the USB drive is 771332 KB
--- Required free space for installer: 42624 KB
--- Copying boot files to the USB drive...
--- Cleaning up the staging area...
--- Making the USB drive '/dev/sdd' bootable...
*** usbimg2disk.sh FAILED at line 471 ***
--- Cleaning up the staging area...
umount2: No such file or directory
umount: /mnt/img.fVZZE0: not found
bash-4.2#

again get strange error.
in that stage i be very bored, and want to simply install bootimage, as so i get the same script from slackware64 14.1 and copy in the same place, and run:

bash-4.2# sh 14.1_usbimg2disk.sh -i usb
usbboot.img usbboot.img.mirrorlist usbimg2disk.sh usbimg2disk.sh.mirrorlist
bash-4.2# sh 14.1_usbimg2disk.sh -i usbboot.img -o /dev/sdd

# We are going to use this device - '/dev/sdd':
# Vendor : UFD 2.0
# Model : Silicon-Power8G
# Size : 7681 MB
#
# FDISK OUTPUT:
#
# Disk /dev/sdd: 8054 MB, 8054112256 bytes
# 1 heads, 62 sectors/track, 253720 cylinders, total 15730688 sectors
# Units = sectors of 1 * 512 = 512 bytes
# Sector size (logical/physical): 512 bytes / 512 bytes
# I/O size (minimum/optimal): 512 bytes / 512 bytes
# Disk identifier: 0xc3072e18
#
# Device Boot Start End Blocks Id System
# /dev/sdd1 * 24 15730687 7865332 c W95 FAT32 (LBA)

*** ***
*** If this is the wrong drive, then press CONTROL-C now! ***
*** ***
Or press ENTER to continue:
--- Using current FAT partition label 'SLCK64_CUR'
--- Available free space on the the USB drive is 729312 KB
--- Required free space for installer: 42624 KB
--- Your USB drive contains directory 'syslinux'
--- This will be overwritten. Press CTRL-C to abort now!
Or press ENTER to continue:
--- Copying boot files to the USB drive...
--- Cleaning up the staging area...
--- Making the USB drive '/dev/sdd' bootable...
bash-4.2#

as you can see, with script from slack 14.1 all be as it must be.
looks like someone is seriously broken usbimg2disk script in slackware64 current.
i can say - it is ok - broken in trying to get something in script better, but looks like, no one is even testing script after make changes in it, and put that one not-working in to distribution?

it looks like a "redhat", "microsoft" and so way, no slackware way...?

Tonus · 11-08-2015, 06:38 AM

Good you noticed the error.

By the way, I wouldn't expect slackware - current to be as stable and tested as stable release is... Would you?

WiseDraco · 11-08-2015, 06:44 AM

Quote:

Originally Posted by Tonus

Good you noticed the error.

By the way, I wouldn't expect slackware - current to be as stable and tested as stable release is... Would you?

stability of slackware become a legendary

and from another side - there is more than 2 years without new slack release, as so, i expect a slack current as a everyday stable release. i think, not only i...

Tonus · 11-08-2015, 09:17 AM

No, I run current for my daily driver but I am aware of something might be broken or might break.

Btw, the current script you used... Was it from a slackware64-current repo? Not sure while reading your post...

Didier Spaier · 11-08-2015, 10:44 AM

Code:

/archives/versions$ diff -u slackware-14.1/usb-and-pxe-installers/usbimg2disk.sh slackware-current/usb-and-pxe-installers/usbimg2disk.sh
--- slackware-14.1/usb-and-pxe-installers/usbimg2disk.sh        2013-09-25 07:14:29.000000000 +0200
+++ slackware-current/usb-and-pxe-installers/usbimg2disk.sh     2015-04-08 00:34:50.000000000 +0200
@@ -148,7 +148,7 @@
   fi
 
   # Set the bootable flag for the first partition:
-  /sbin/sfdisk -A $USBDRV 1
+  echo ',,,*' | /sbin/sfdisk $USBDRV -N1
 }
 
 # Parse the commandline parameters:
/archives/versions$

This changes the line #151 so I don't see how that could change the behavior in line #363. I would rather look into the format of fdisk or sfdisk's output or something like that, but I will make some tests to check. Stay tuned.

Alien Bob · 11-08-2015, 10:54 AM

Quote:

Originally Posted by Didier Spaier

Code:

/archives/versions$ diff -u slackware-14.1/usb-and-pxe-installers/usbimg2disk.sh slackware-current/usb-and-pxe-installers/usbimg2disk.sh
--- slackware-14.1/usb-and-pxe-installers/usbimg2disk.sh        2013-09-25 07:14:29.000000000 +0200
+++ slackware-current/usb-and-pxe-installers/usbimg2disk.sh     2015-04-08 00:34:50.000000000 +0200
@@ -148,7 +148,7 @@
   fi
 
   # Set the bootable flag for the first partition:
-  /sbin/sfdisk -A $USBDRV 1
+  echo ',,,*' | /sbin/sfdisk $USBDRV -N1
 }
 
 # Parse the commandline parameters:
/archives/versions$

This changes the line #151 so I don't see how that could change the behavior in line #363. I would rather look into the format of fdisk or sfdisk's output or something like that, but I will make some tests to check. Stay tuned.

That changed line is part of the "makebootable" function which is called on line 471. That seems to generate his error:

Code:

--- Making the USB drive '/dev/sdd' bootable...
*** usbimg2disk.sh FAILED at line 471 ***

His other error:

Code:

usbimg2disk.sh: line 363: UFD: command not found
*** usbimg2disk.sh FAILED at line 363 ***

seems to be caused by a space in the original FAT label of the stick. I am pretty certain that the original label contained a SPACE character, something like "UFD stick". The "lsusb" output mentions "UFD" as the vendor of the stick.

I sent Pat a patch that changes the script to allow spaces in the USB stick's FAT label:

Code:

--- usb-and-pxe-installers/usbimg2disk.sh	2015-04-08 00:34:50.000000000 +0200
+++ /home/eha/work/curdev/usbimg2disk.sh	2015-11-08 15:33:05.540959346 +0100
@@ -134,7 +134,7 @@
   # It will enable the installer to mount the fat partition automatically
   # and pre-fill the correct pathname for the "SOURCE" dialog.
   # Format with a vfat filesystem:
-  /sbin/mkdosfs -F32 -n ${THELABEL} ${TOWIPE}1
+  /sbin/mkdosfs -F32 -n "${THELABEL}" ${TOWIPE}1
 }
 
 makebootable() {
@@ -345,7 +345,7 @@
     echo "--- Last chance! Press CTRL-C to abort!"
     read -p "Or press ENTER to continue: " JUNK
   fi
-  ( reformat $TARGET ${FATLABEL} ) 1>>$LOGFILE 2>&1
+  ( reformat $TARGET "${FATLABEL}" ) 1>>$LOGFILE 2>&1
 else
   # We do not format the drive, but apply a FAT label if required.
 
@@ -358,12 +358,10 @@
     # User gave us a FAT label to use, so we will force that upon the drive:
     echo "--- Setting FAT partition label to '$FATLABEL'"
     MTOOLSRC=$MTOOLSRCFILE mlabel s:${FATLABEL}
-  elif [ -n "$(/sbin/blkid -t TYPE=vfat -s LABEL -o export $TARGETPART)" ] ; then
+  elif [ -n "$(/sbin/blkid -t TYPE=vfat -s LABEL -o value $TARGETPART)" ] ; then
     # User did not care, but the USB stick has a FAT label that we will use:
-    eval $(/sbin/blkid -t TYPE=vfat -s LABEL -o export $TARGETPART)
-    FATLABEL=$LABEL
+    FATLABEL="$(/sbin/blkid -t TYPE=vfat -s LABEL -o value $TARGETPART)"
     echo "--- Using current FAT partition label '$FATLABEL'"
-    unset LABEL
   else
     # No user-supplied label, nor a drive label present. We apply our default:
     echo "--- Setting FAT partition label to '$FATLABEL'"

I have no idea why the OP stumbles over the new sfdisk command.

Didier Spaier · 11-08-2015, 03:06 PM

Tried this

Code:

bash-4.2# LANG=en_US.utf8 ./usbimg2disk.sh -f -i usbboot.img -o /dev/sdc

Failed from this line (added an echo to debug):

Code:

mount -o loop,ro /archives/versions/slackware64-current/usb-and-pxe-installers/usbboot.img /mnt/img.Vkxc0v
mount: wrong fs type, bad option, bad superblock on /dev/loop0,
       missing codepage or helper program, or other error
       In some cases useful info is found in syslog - try
       dmesg | tail  or so

*** ./usbimg2disk.sh FAILED at line 407 ***

But wait! dmesg|tail says:

Code:

[17831.203383] FAT-fs (loop0): codepage cp437 not found

This is with a custom (probably too much stripped) kernel. I will retry in a clean VM.

Didier Spaier · 11-08-2015, 04:17 PM

Not surprisingly the same command worked in a VM!

There is of course no chance to mount a FAT partition without the support for cp437... Lesson learned.

But the bug that prevents booting in UEFI mode is still there. Output of "tree /mnt":

Code:

/mnt
└── syslinux
    ├── EFI
    │   └── BOOT
    │       ├── BOOTX64.EFI
    │       ├── elilo.conf
    │       └── message.txt
    ├── f2.txt
    ├── huge.s
    ├── initrd.img
    ├── ldlinux.sys
    ├── memtest
    ├── message.txt
    ├── setpkg
    └── syslinux.cfg

3 directories, 11 files

The EFI partition should be directly at the root of the tree, not under syslinux. The current UEFI spec says:

Quote:

3.5.1.1 Removable Media Boot Behavior
On a removable media device it is not possible for the FilePath to contain a file name, including sub directories. FilePathList[0] is stored in non volatile memory in the platform and cannot possibly be kept in sync with a media that can change at any time. A FilePathList[0] for a removable media device will point to a device that supports the EFI_SIMPLE_FILE_SYSTEM_PROTOCOL or EFI_BLOCK_IO_PROTOCOL. The FilePathList[0] will not contain a file name or sub directories.

If FilePathList[0] points to a device that supports the EFI_SIMPLE_FILE_SYSTEM_PROTOCOL, then the system firmware will attempt to boot from a removable media FilePathList[0] by adding a default file name in the form \EFI\BOOT\BOOT{machine type short-name}.EFI. Where machine type short-name defines a PE32+ image format architecture. Each file only contains one UEFI image type, and a system may support booting from one or more images types

So, the firmware should find /EFI/BOOT/BOOTX64.EFI but is not required to find /syslinux/EFI/BOOT/BOOTX64.EFI.

I checked that again as my laptop's firmware is easy to set up: in UEFI only it doesn't find the EFI image, in UEFI+Legacy with priority UEFI it boots with syslinux.

Eric, I emailed you then Pat about that on 20/11/2013 and the same day Pat answered "I'll have a look at the script and see what the best approach for it is. However, I'd like to note that there's actually little use for the script any more -- the DVD ISO can be copied directly to a USB stick using dd (with bs=1M for best write performance), and will boot and install from either BIOS or UEFI."

Maybe at this point in the development cycle it's time to decide either way: fix, or do not ship the script in next release.

I attach the already sent patch (and will apply it again on -current to be 101% sure it still works).

Didier Spaier · 11-08-2015, 05:01 PM

Confirmation: once the patch is applied in UEFI mode only or with priority UEFI the ELILO greeting screen appears, else the SYSLINUX one.
Now the tree is as follows:

Code:

bash-4.2# tree /mnt
/mnt
├── EFI
│   └── BOOT
│       ├── BOOTX64.EFI
│       ├── elilo.conf
│       └── message.txt
├── f2.txt
├── huge.s
├── initrd.img
├── ldlinux.sys
├── memtest
├── message.txt
├── setpkg
└── syslinux.cfg

2 directories, 11 files
bash-4.2#

Tested on an up to date clean Slackware64-current.

laprjns · 11-09-2015, 04:51 AM

Quote:

Originally Posted by Didier Spaier

The EFI partition should be directly at the root of the tree, not under syslinux

I ran into this over the weekend and left a comment on Alien Bob's blog
http://alien.slackbook.org/blog/inst...b-thumb-drive/

Quote:

Originally Posted by Didier Spaier

... the DVD ISO can be copied directly to a USB stick using dd (with bs=1M for best write performance), and will boot and install from either BIOS or UEFI."

I'm certainly no expert here, but after thinking about this, maybe the problem is with the structure of the installer image (usbboot.img)and not with the script. After all, the script is just coping the image as is.

Alien Bob · 11-09-2015, 05:15 AM

Quote:

Originally Posted by laprjns

I ran into this over the weekend and left a comment on Alien Bob's blog
http://alien.slackbook.org/blog/inst...b-thumb-drive/

I'm certainly no expert here, but after thinking about this, maybe the problem is with the structure of the installer image (usbboot.img)and not with the script. After all, the script is just coping the image as is.

No, the script is capable of doing a lot more than just copying the content of the img file to a USB stick.
It is correct that since Slackware 14.1 the official ISO images are hybrid, and when you "dd" the DVD ISO image to a USB stick, you have a bootable Slackware installation medium with nothing else required.
The usbimg2disk.sh script is still useful if you have a USB stick with existing data on it that you do not want to erase. The script can make the USB stick into a bootable Slackware installer without touching the existing files.

Pat told me he would apply both my "label with spaces" patch (which he already did in our private repository) as well as Didier's patch that fixes UEFI boot.

Didier Spaier · 11-09-2015, 05:29 AM

... And if we "mount usbboot.img /mnt" we see exactly the same tree as in post #9.

I am happy with the decision as I think that this script is still useful, for the use case Eric listed and also as some people still own USB keys with a size of 2G or less.

I take this occasion to also thank Eric for another script: gen_repos_files.sh, that comes very handy for anyone maintaining a repository of third-party packages.

laprjns · 11-09-2015, 05:42 AM

Quote:

Originally Posted by Alien Bob

No, the script is capable of doing a lot more than just copying the content of the img file to a USB stick.

My apologies, didn't mean to imply that that was all the script did, but rather I meant that when it came to the particular function of transferring the installer image all it did was copy it as is. But after taking a closure look at Didier script I now realize that it wasn't just just copying.

Quote:

Originally Posted by laprjns

I'm certainly no expert here

I guess this was redundant.

WiseDraco · 11-09-2015, 10:45 AM

Quote:

Originally Posted by Tonus

No, I run current for my daily driver but I am aware of something might be broken or might break.

Btw, the current script you used... Was it from a slackware64-current repo? Not sure while reading your post...

bash-4.2# ls -la
total 42832
drwxr-xr-x 2 drago users 4096 Nov 8 13:29 .
drwxrwxrwx 23 root root 4096 Nov 8 13:09 ..
-rw-r--r-- 1 root root 15697 Sep 25 2013 14.1_usbimg2disk.sh
-rw-r--r-- 1 drago users 31203 Nov 8 13:03 README_PXE.TXT
-rw-r--r-- 1 drago users 20952 Nov 8 13:03 README_PXE.TXT.mirrorlist
-rw-r--r-- 1 drago users 9197 Nov 8 13:03 README_USB.TXT
-rw-r--r-- 1 drago users 20951 Nov 8 13:03 README_USB.TXT.mirrorlist
-rw-r--r-- 1 drago users 574 Mar 27 2013 pxelinux.cfg_default
-rw-r--r-- 1 drago users 21868 Nov 8 13:03 pxelinux.cfg_default.mirrorlist
-rw-r--r-- 1 drago users 43643904 Nov 8 13:04 usbboot.img
-rw-r--r-- 1 drago users 20496 Nov 8 13:04 usbboot.img.mirrorlist
-rw-r--r-- 1 drago users 15710 Nov 8 13:04 usbimg2disk.sh
-rw-r--r-- 1 drago users 20952 Nov 8 13:04 usbimg2disk.sh.mirrorlist
bash-4.2#

slackware64-current downloaded yesterday.

ckrzen · 11-30-2015, 03:14 PM

BUG-REPORT: usbimg2disk.sh from slackware64-current as of 11/30/2015

1. usbimg2disk.sh: grub.cfg points to the initrd.img under the isolinux/ dir after creation, where it _does_ exist in my slackware64-current/isolinux/ tree. It is not being moved to the usb stick during the cp command(I suppose).

REMEDY: I just mounted / edited the grub.cfg in the EFI/BOOT/ dir on the usbstick to point to the: /EFI/BOOT/initrd.img and was able to restart the installer successfully.

2. usbimg2disk.sh: I only rsync the binary trees from the mirrors so, the /src/ dir does not exist for the usbimg2disk script to use. The usbimg2disk script _is_ smart enough to ignore the java link under the /extras/ dir at line #289. It needs to also ignore the 'flashplayer-plugin" link.

REMEDY: # rm ../extra/flashplayer-plugin # before running the usbimg2disk script.

3. Login MOTD in the installer initrd.img: it suggests using 'cgdisk' or 'gdisk' on UEFI / GPT partitions, however, the version of 'cfdisk' and 'fdisk' in the -current trees are GPT-aware and useful for partitioning as well, now.