[SOLVED] Problem using iso2usb.sh LUKS encryption

business_kid · 09-18-2019, 11:51 AM

I'm having a problem, in that this freezes at a point. The processes in top show a few more kworker jobs than I'm used to, and usb-storage, but nothing's working. I'm booting on the 4.19.59*huge kernel, have it's modules installed, etc, and have cryptsetup, & squashfs. Nothing is complaining to ldd about lack of support. Here's the error:

Code:

mount: /mnt/alieniso.8fGU7K: WARNING: device write-protected, mounted read-only.
147501 blocks
--- Copying files from ISO to USB... takes some time.
    732,273,237 100%  136.68MB/s    0:00:05 (xfr#75, to-chk=0/83)
--- Creating 1024 MB container file using 'dd if=/dev/urandom', patience please...
0+0 records in
0+0 records out
0 bytes copied, 0.00404137 s, 0.0 kB/s
--- Encrypting the container file with LUKS; enter 'YES' and a passphrase...

WARNING!
========
This will overwrite data on /dev/loop1 irrevocably.

Are you sure? (Type uppercase yes): YES
Enter passphrase:
Verify passphrase:
--- Unlocking the LUKS container requires your passphrase again...
Enter passphrase for /mnt/alienusb.WmxCPt/slhome.img:

# FROZEN HERE AND NEVER PROGRESSES. Processes die away to 0%.

I have entered the passphrase. If I enter a wrong password, it makes me do it over, so I don't think the script is hanging. I have every compression algorithm except one (ztsd?) which I never heard of. If I do it without the encrypted home, iso2usb.sh is fine.
My line was: iso2usb.sh -i slackware64-live-xfce-current.iso -o /dev/sdb -c 1G -w 10

Ctrl_C doesn't kill it. I have to do a kill -9 on the appropriate bash terminal. 'ps -e |grep tty3' reports 2 bash processes and cryptsetup. I presume it's cryptsetup that has diminished interest. Sending a HUP to cryptsetup gives me an offer to press Ctrl_C to abort. But it doesn't throw an error or restart work.

Alien Bob · 09-19-2019, 10:50 AM

How long did you wait? The script first creates the 1 GB container in a 'fast' way, but then after the container is LUKS-formatted, a second 'dd' command will write all zeros to the container, and writing 1 GB of zeros to a LUKS device may take some time.

business_kid · 09-20-2019, 05:04 AM

I left it running several hours - most of the day; Around 6 hours was the longest time, because I figured if it couldn't rewrite home in that, it was farcically slow. When installing the squashfs, it seemed farcically slow. It did most of the first gig fast, then slowed so 500kB - 2MB/S.But even at 1MB/S, it should complete in 3-4 hours. All the processes (cryptsetup, & usb-storage) which showed on top initially were sitting back with their feet up instead of running, so it was pretty clear it wasn't going to complete. I tried giving cryptsetup a HUP signal, but after 6 hours it only offered me a 'quit' option.

EDIT: BTW, this wasn't 1G, this was 10G for /home. This is my first try at rolling what I want. There's a program I run under wine which is ≥5G atm, and growing by 500K - 1G every month or two and I want it on this thing. I left it at 10G to allow for updates.

syg00 · 09-20-2019, 05:19 AM

Read this - well-known problem. There are mitigations, some mentioned in the article.

Alien Bob · 09-20-2019, 07:13 AM

I tried yesterday on a 32 GB USB stick with the command

Code:

./iso2usb.sh -s -i slackware64-live-xfce-current-iso -c 30% -C 30%

which means the script created two LUKS containers of roughly 10 GB, one for /home and one to store the persistent data.
That obviously took a while to complete but within the hour, and the LED on the USB stick that shows activity was blinking all the time.
It may indeed be an issue with your own USB hardware like the previous poster indicated.

Firerat · 09-20-2019, 07:25 AM

to see if dd is progressing

from dd manpage

Code:

       Sending a USR1 signal to a running 'dd' process makes it print I/O statistics to  standard
       error and then resume copying.

business_kid · 09-20-2019, 11:51 AM

@Fireat & Alien_Bob: Thanks for the useful ideas & feedback. A USR1 signal, eh? I've no led on my usb stick, unfortunately, and may be stuck on usb-2.0. My usb key is a few years old. Hdparm reports 21MB/Sec.

@syg00: Very interesting link taking me into kernel entrails. I don't have a 32bit Kernel lying about, but I do have Multilib, and I suppose I could just d/l one from one of the slackware mirrors. For the record, I have 6G of memory, and 6G of swap, which is a little OTT for what I and my cpu does. But it could build up a lot of dirty_bytes! From my recent 'Hibernation' thread, It emerged I apparently haven't used any swap in the last 2 years.

Could I get out of this by issuing regular 'sync' commands?

/goes off to re-read that link and cobble up reasonable numbers to throw at /proc/sys/vm

/Later: I was set @

dirty_background_ratio=10%
dirty_ratio=20%

I halved them (5% & 10%), and saw the processes more often (initially mkfs.ntfs & usb-storage for a slightly adjusted iso2usb.sh which allows space for a 4th partition). Then I upped them (20% & 30%), and things ground to a halt. I tried a sync, but sync won't complete. I conclude there is very slow talking out to that usb drive. It's written 10% of 9.8G while I've been slowly typing this paragraph.

EDIT: On mke2fs, it's actually doing 26 seconds per 1% (of 9.8G which translates to a couple of Megs/S(?). I won't hold my breath. A sync never completes. So the bottleneck is the usb drive. The lower numbers seem to go faster.

And I've a filesystem error on /dev/sdb3, which I've just made yesterday with iso2usb.sh. I'll go off and waste time on that now…

Firerat · 09-20-2019, 12:02 PM

if its and old usbstick it may be on its last legs

you could chunk up the dd, with a until loop using a seek
throw a printf progress in there maybe with a sync
capture time per loop and work out an ETA

business_kid · 09-20-2019, 01:38 PM

Quote:

Originally Posted by Firerat

if its and old usbstick it may be on its last legs

you could chunk up the dd, with a until loop using a seek
throw a printf progress in there maybe with a sync
capture time per loop and work out an ETA

Most of that went over my head - after the 'dd' bit. The most I ever used my elementary C knowledge for was to program tiny PIC micro-controllers, and fix the syntax on a patch for the radeon driver (much to the dev's disgust, whose blushes I will spare). You've obviously got a clever script idea there, but I don't know if it's needed. I have a 1.5G iso with my mods (which will probably puke anyhow). The iso2usb gives a progress report. I write most of the first Gig very fast. Then it slows to ~1MB/S, and takes much longer to get from 900K - 1G, and on to 1.5G. I'm running badblocks -vw on ~19G, which it has thrown out with an error.

So this is solved, because my usb disk was a dud.

business_kid · 09-21-2019, 12:31 PM

Actually the badblocks was error free. The thing I thought was an error was an 'interruption,' which killed the process.

Well, I ran line 4 times in succession with a semi-colon between them

Code:

time dd if=/dev/urandom of=/dev/sdb3 bs=1M count=500 & time sync

The dd output gave optimistic figures but the syncs brought reality. My times with successive copies get slower & slower and the second sync doesn't complete; rather, it completes all right, but the prompt never returns to the terminal

So, my real disk is an ssd ~450MB/S on a slow day. I have sdb1 & sdb2 written OK to the usb, and partitioning done. If all else fails, I can buy a usb 3.x usb key, loop mount the iso, assemble this on a hard disk then copy it over.