Hi,

Using dd:

I am trying to copy a slightly older 32GB USB 3.0 SanDisk Extreme Thumb drive to another, newer 32GB USB 3.0 SanDisk Ultra Thumb drive.
I am getting the copy to run almost to completion, but then get a "no space left on device" error.

Lamentably, the target drive is smaller!

So I see that in parted the full device partition size for the source drive,(let's say it comes up as "/dev/sdb") is 32GB while the other (let's say "dev/sdc") is 31GB.

I have tried a tip from another forum to set a count based on the smaller drive's size in bytes. So for example, running dd like this:

dd bs=4096 if=/dev/sdb of=/dev/sdc count=$(blockdev --getsize64 /dev/sdc).

I have even modified that command a bit to subtract up to 1GB from the count, thinking that would give plenty of margin of safety and not overrun the device. So for example,

dd bs=4096 if=/dev/sdb of=/dev/sdc count=`echo "$(blocksize --getsize64 /dev/sdc)-1073741824"|bc`

A bit of command substitution that knocks the size returned from blockdev down by that many bytes.
I have tried downsizing the source drive's logical volume and ext4 filesystem. It didn't work

I'm wondering if I have to concentrate on the source drive and resizing its partitions down from 32GB?

The source drive has an encrypted vg, lv, an installation of Ubuntu and work that I've saved.
I could build the target drive up the same way, but it takes hours. dd would be perfect.
It does work to some extent, as I can boot the target USB after the copy. It just won't open the encrypted vg correctly, as it senses an overrun device.

Thanks ahead for any ideas.

Brian

if you are copying partition you can resize it down, so it will fit on another drive or just buy 64gb drive if that is an option.

try booting the source drive with gparted live-usb and shrink it down a tad.

Technically, your thread title should be "dd: can't copy a 32GB USB 3.0 SanDisk extreme to a 31GB USB 3.0 SanDisk Ultra". This is not a problem with dd.

I would recommend getting a larger target drive if you don't want to build the target from scratch. dd just can't fit more bytes onto a device than that device can hold.

You could try wiping out your target drives partition table (use dd for this), and then repartition it with only a single partition. Could be that the target drive actually has more than one partition on it currently due to some new encryption or plug-n-play stuff that SanDisk may have included. Wipe all that out (if it exists) and create one single partition. Then maybe the contents of your old drive will fit. Maybe.

We kind of assume you have a linux file system and not a windows file system on this source usb.

Might be possible to use gparted to copy the files. http://gparted.sourceforge.net/displ...aste-partition

As noted the issue is trying to use a bit by bit copy program where the original has more bits. It leaves you with too many bits.

There are some other ways to copy also that may work. tar, rsync, cpio and other tools like partimage if it takes file by file.

You might be able to just fix the boot with syslinux or grub and then copy files. Also some references in grub or boot may need to be changed no matter what you do.


Pretty sure Acronis can do it and maybe other ghost type apps that support linux file systems.

A big problem with your effort thus far is that you are computing a size in bytes which dd is taking as a count of 4096-byte blocks.

Maybe your target drive is a U3 drive, which Sandisk commonly put on thumb drives for a while. U3 messes with partitioning on the drive and creates a hidden partition. If that's the case, the only way I've found to remove U3 is visit the sandisk website and download and run their u3 removal tool.

I bought a 3pack of sandisk thumb drives at Costco a couple years ago and found that they were all u3 enabled.

Thanks for the ideas.
The source drive is Ubuntu Linux with 1 vg & 1 lv.

I am inclined to think as others have mentioned, that the target drive- even though it's a speced 32GB just like the source- has some sort of limitation(ie- hidden partition) on allocating all the physical extents, and therefore can't offer its full 32GB.

Is there a linux tool, command or utility to blow away any possible hidden partition?
Along this line, I believe I tried (from the other forum) running blank input option to dd:

dd bs=1 if=/dev/zero (or/dev/urandom) of=/dev/sdc
Apparently this would wipe anything from beginning to end, and then a full partition could be laid down.
I think I had tried that, and it still came up 1GB short of its speced size. I'll try again.

The idea of dd is nice because it's a "capture-all" affair and one doesn't have to surgically copy in 1 layer at a time above it.

You don't need to wipe the whole thing, and not with random data either - if you are just wanting to reuse it yourself. If you're giving it away to someone else, and it used to contain sensitive data, yes, a more secure and full wipe is needed.

dd bs=512 count=1 if=/dev/zero of=/dev/yourdevice

...should do it to zero out the MBR of a harddisk. Change count=1 to count=63 if you want to zero out the entire "track 0"

As far as a SanDisk USB stick, with U3 or who-knows-what-else they do to it, I'm not sure exactly where and how much to zero out to allow you to repartition it back to its max capacity. I would imagine a little Google searching for "restore SanDisk U3 device to max capacity" would generate a few hits. There's got to be a lot of other people who want to (or did) do this exact thing. Add "linux" or "dd" to that search string to maybe target results more to what you want (i.e., avoiding Windows solutions).

For drives with a legacy DOS partition table, yes, and USB flash drives should be in that category. But, for large hard disks partitioned with GPT, you also need to clear out the GPT partition table header at LBA 1 as well as the backup GPT header at the very end of the disk, or else GPT-aware partitioning tools will still see the old partition structure.

That's why I said "MBR". Maybe that wasn't the correct term (I see you used "legacy DOS partition table" - I thought "MBR" was synonymous with this, but I may be mistaken).

I don't have any GPT systems and don't know the procedure to zero them out for a clean start. I would like to learn the specifics of how to do that some day, but I don't want to hijack this thread. We're talking specifically about USB thumbdrives here, and AFAIK, they all use "MBR", or "legacy DOS partition table", or whatever is the technically correct way to describe this partitioning scheme. I've never looked into how "U3" is implemented. I guess I always just assumed is was simply a special second partition in the "MBR" partitioning scheme with special boot code. But maybe not...

For the purposes of this discussion, "MBR" and "legacy DOS partition table" are basically the same thing (Technically, the DOS partition table occupies part of the sector that holds the MBR.), and I did agree that USB flash drives would very likely be partitioned that way. But, you did say, "zero out the MBR of a harddisk," and I thought it prudent to mention that for some hard disks this would not be sufficient.

U3 is a good deal more complicated than just a 2nd partition. The device presents itself to the system as a USB hub with two drives. Removal of the U3 feature requires special software to reconfigure the device. There is some basic information on Wikipedia.

Question is:
Could dd be blocked for overwriting this 2nd, hidden partition component of a U3 drive?
I've run dd if=/dev/zero (and /dev/urandom) of=/dev/sdb bs= 1 (and 4096 and 8192).
In every case, the size of the resulting USB device in parted is reflected as 31GB.
Could 1GB worth of a hidden partition be escaping the "smashing steamroller" of dd?
If that's the case, I guess I do need to spend time trying to find and eliminate that partition with other tools.

Called SanDisk Tech Support.
Looks like they think the drive is defective.
They'll send a swap-out replacement.
He had me quick format in Windows for NTFS, and it was showing 1/2 a GB less than they expected.

Thanks all.

Brian

Thanks for the update.