Check your BIOS for the default boot device sequence.
BTW Have you investigated the reason for the poor clone speed?

I do agree that re-installing and re-configuring applications is a nuisance and a major downtime. Which is why snapshots is good for this purpose. You can use snapshots for backing up filesystems and testing application upgrades. And if things brakes, you can revert back to the last working state of the system or restore from backup.

I'm thinking of going back to using LVM and snapshots as it is more easier to restore a root filesystem back to a working state as to starting from scratch. Right now, I am refreshing my LVM skills since I done this a long time ago.

There's some interesting thoughts there. Just bear with me if I have difficulty in remaining afloat at times. I'll try to keep up but 'newbie' means that I do not yet have much Linux experience or knowledge on which to draw.

The idea that the boot sequence may be having difficulty in identifying the correct disk sounds plausible. When I read the clone report I was not surprised to find that the source and target disks were identical. That is, after all, the definition of cloning but I was a little concerned to note that both disks carried the same identifier – 0x000934F6. I did wonder how the OS would distinguish between them. Perhaps it can't. I'll change the volume label on the cloned disk and see if that resolves the issue – but first I will have to take another look at the user guides to work out how to do it. (Newbie, remember.)

The question as to whether I have investigated the reason for the very slow cloning rate is one that I cannot yet answer, for the reason I have just given. I haven't a clue how to do it. From an earlier response, if I have understood it correctly, it would appear that, without an additional option, the basic 'dd' command performs a bit copy, which, I can appreciate, is likely to be slow. If I add the 'bs' option, as suggested, and this converts the bit copy into a byte copy then that might indeed be faster, raising the further question of whether additional, or other, options would reduce the copy time even more.

I have read the syntax of the 'dd' command but with limited comprehension. If anyone can suggest some useful alternative approaches I will be happy to try them. I would like to get down to less than 30 minutes for the 80Gb clone and that might prove possible. USB 3.0 might enable me to do even better but I don't have USB 3.0 on my computer which is of very early 2007 vintage. One thing to bear in mind is that 'dd' works, despite its limitations, whereas other disk copying/cloning utilities have failed. It could be a case of the bird in the hand being worth two in the bush.

To quote Douglas McArthur, “I will return,” with good news or bad but I am awaiting the delivery of a couple more USB disk caddies and it may be a day or two before I can try further 'dd' options. With my first and only disk back-up 'in the can' I want to avoid doing anything that will jeopardise it. Hopefully changing the volume label will resolve the possible disk misidentification issue when booting but I will have to do a bit of reading before I attempt to change the boot sequence, if that proves necessary.

If you are using dd, don't let it use the default buffer size - that causes a lot of delay on either input or output. use "bs=1M" or so (you can experiment with the size).

1MB buffers allows a lot of data to read at once (which is pretty quick for a hard disk), then it will saturate the USB - allowing the system to always keep the USB busy. Using the default (512 bytes) causes the USB to go idle - a LOT.

1 members found this post helpful.

If you are using dd, don't let it use the default buffer size - that causes a lot of delay on either input or output. use "bs=1M" or so (you can experiment with the size).

1MB buffers allows a lot of data to read at once (which is pretty quick for a hard disk), then it will saturate the USB - allowing the system to always keep the USB busy. Using the default (512 bytes) causes the USB to go idle - a LOT.

One way to just get a rough idea is to use "time". The output will be how long it took to copy the data. Both elapsed time as well as user mode time and system mode time. As the buffers get larger (going from 512bytes and up) you should see the elapsed time go down but the system time will increase (more buffers to handle).

1 members found this post helpful.

Before confronting the arcane topic of changing volume name/UUIDs I thought that I would follow Mr. Occam's advice and first check that my earlier statements were correct. Not all of them were. If the external USB hard disk is mounted when the computer is booted then the boot takes place from the USB drive and not from the on-board SATA drive as demanded by the boot sequence:-
1. Onboard or USB floppy drive (declared 'not present')
2. Onboard SATA hard drive
3. Onboard IDE hard drive (declared 'not present')
4. Onboard USB CD ROM
5. Onboard network controller (declared 'not present')
6. USB device (declared 'not present')
If the external drive is not mounted then the computer boots from the onboard SATA hard drive, in the usual way. I should have checked this more carefully but I still find it odd that the computer should boot from a device different from that specified in the boot sequence – i.e. appears unable to distinguish between an on-board SATA and an external USB drive. Hopefully I shall not need to investigate this further. The solution, as I was advized, is to make sure that the USB drive is unmounted before shutting down.

I thought about the suggestion of changing the volume name/UUID but, whilst I don't understand this at all well, I have to ask whether it would have worked. If the boot sequence effectively uses this parameter to identify the boot device then if I changed it on the cloned disk would the latter then be recognized as the appropriate boot disk if it were exchanged for the existing main drive? If not, the value of cloning as a back-up strategy could be compromised.

Leaving this aside, as probably being no longer relevant, I have to say that things are now much clearer. My misunderstandings and silly mistakes were kindly corrected (it was worth forgoing the cigar!) and Mr. Occam was once again vindicated. No commercial software is needed to clone a hard disk. 'dd' does the job nicely and the simplest suggested form of the mount command, namely 'mount /dev/sdb1 /mnt/xdisk', works just fine, making all the cloned files viewable via the GUI, should individual file inspection or restoration be more appropriate than replacement of the whole main hard drive. 'umount /dev/sb1' in turn unmounts the external drive before shutting down. Just three simple commands. I have no doubt that the relative merits of cloning versus mirroring, or backing up in some other way, will continue to be debated, but this does it for me. My thanks to all concerned for their help.

I still have the slow cloning speed to pursue but my caddies have arrived and I now understand what the 'bs' operand does in the 'dd' command. I'll report the results in due course before marking this thread as 'Solved'.

By "not mounted" I assume you mean "not installed". There are no mounts at the beginning of boot.
The problem is the way Linux identifies disks. What happens is a scan across all controllers in parallel. This means that sda is identified as the FIRST disk that spins up (which is usually the /boot volumn). The second disk (sdb) can be ANY disk that is attached - whether it is USB or SATA or whatever. I have one case (SATA) with 4 drives. sda has the boot, sdb is USUALLY the home disk... but if I have the backup disk installed, then that is sdb instead, and the home disk is sdc. It all depends on what disk spins up first. In the case of two equal drives... it becomes random.

For this reason Linux no longer uses device names to choose what to mount. Instead it uses either volume labels (in my case root0 or root1) or use UUID strings (only choice for swap). Of course it fails if I happen to have two labels the same, or two UUID strings the same. You can't tell WHICH device will actually be used - it depends on the order of the list.
Cloning with dd works acceptably well. Some of the fancier cloning utilities may change the UUID and change the specified mounts in the /etc/fstab, initrd and boot menues (not having used them, I prefer to use tar files as it doesn't care about volume labels, though I do have to change the fstab/initrd/boot menu manually).

This doesn't matter if the disks are actually going to be exchanged - but it does matter if both are installed during boot... Some of the problems that occur happen with LVM - if there are two disks identified identically then LVM can't build a raid set as it can't tell which ones should be used...
One additional reference (though you might have found this): https://wiki.archlinux.org/index.php/disk_cloning

1 members found this post helpful.

I find that (commonly) sda is the disk that is connected to the SATA 1 connection on the motherboard,
similarly sdb corresponds to SATA 2.

But I wouldn't absolutely rely on it.

1 members found this post helpful.

Usually the boot volume is on the disk that is spun up as that is the one the BIOS/UFI references to load the boot from, so it is already spun up. If you swap the controller it will still be sda...

1 members found this post helpful.

Just when I thought that I understood the game, someone moved the goal posts - for the best possible reasons of course - but I think that I have got it now.

Which is where I misconstrued the result in assuming that I was witnessing cause and effect. In fact, things are a lot simpler than I originally imagined. The computer does not need to be rebooted nor the USB disk connected at this point. If the commands are issued correctly (another regrettable cause for confusion) the USB disk can be connected at any time after boot meaning, of course, that the computer will not boot from it - even though it was heartening to observe that it could be booted from the external disk under the appropriate circumstances.

There was also confusion, at first, in distinguishing between the situation when a blank disk was connected to the USB port and that when the subsequently cloned disk was attached. In the former case, whilst 'fdisk' readily 'saw' the newly-attached external disk, the latter could not, of course, be mounted, whereas, once cloned it could be and then viewed via the GUI. That all seems pretty obvious now but, if you have never done it before and have come from a Windows XP background, where you rarely, if ever, had to 'mount' anything, it can be rather daunting, particularly if you throw in a few syntax errors to make matters worse.

All now appears a lot clearer. There was, however, one rather puzzling feature that now appears more readily understandable. As explained, when the external disk is attached its 'address' can be unpredictable. During my first attempts to connect it, it appeared in /dev as /sdb. Now, /sdb has disappeared, to be replaced by /sdc. I was initially worried that the system would continue to work its way through the alphabet but thankfully it has continued to use /sdc irrespective of which of the three USB disks is attached.

Now to the cloning speed issue:-
No, I had not seen this before. I am grateful that you drew my attention to it. I tried the suggested 'dd' syntax, namely:-
The good news was that the chosen value of bs=1M reduced the cloning time from 4 hours (claimed data transfer rate 5.7 Mbyte/sec) to roughly 45 minutes at a claimed data transfer rate of almost 30 Mbyte/sec.
The bad news was that the operation ended with the error message, "No space left on device x". In this case x=c. This message was reinforced by the information:-
76293+1 records in
76293+0 records out
There appeared to be a one-record discrepancy, rendering the cloned disk unreadable and therefore unusable. I was surprised that this expectedly valid syntax should fail in this way but decided that it was irrelevant in my case. If my clone is not perfect then it is of no use as a back-up. It therefore matters little whether records are padded out with zeroes after an error occurs, in order to maintain synchronization, or not. Any error invalidates the operation and will, I presume, be evident from the record count. So I omitted the 'conv=noerror....' parameter and re-cloned. The result was an error-free exact copy.

The next task was to optimize the value of 'bs'. Having started with the recommended '1M' I next raised this to '2M', which resulted in no further reduction in the clone time or increase in the data transfer rate. Reducing bs to 500K left the data transfer rate unchanged but 'bs=250K' reduced it significantly to about 16Mbyte/sec, yielding a clone time approaching 1.5 hours. 'bs=500K' therefore looks to be the most appropriate setting. It appears unlikely that the 30Mbyte/sec data rate can be improved. If I have uderstood things correctly, that represents about half the maximum speed for a USB 2.0 port. I am quite happy with that.

The final question, I suppose, is whether I should consider any alternatives to 'dd'. One or two options have been suggested and I appreciate that 'dd' might not be the most appropriate tool in all circumstances but in my case it seems entirely suitable. I suggest, therefore, that now is the time to declare this problem solved. My thanks once again goes to those who have helped.

dd block size
Can you try a block size of 512K - it might make no difference - but its a power of 2.

Are you aware thatUSB 2.0 can't go that fast, but you wouldn't expect much better sequential physical disk write speeds on USB 3.0.

Essentially, the alternatives dd as a cloning tool come down to:-
a) Sticking with dd
b) Using a compression tool, so that you can hold several backups on the spare disk.
You would be able to exclude easily regenerated files and temporary stuff from the backup.
c) Periodically do a full save of your data and do differential backups on a daily basis.
This would give you snapshots of your files for changed files for the last couple of weeks
and older (months) versions of less recently changed files.

I use option (c) to a partition on an internal disk which in turn gets backed up to an external disk "periodically".
I exclude nearly all OS files - it's worth backing up things like fstab etc.
I've got quite a lot of HD videos on disk - but they are too much effort to backup.

Just to confuse you I also operate a backup strategy every few months.
I have a 2TB portable USB drive and it's partitioned with LVM.
I copy all files from all partitions, unselectively, for half dozen PC's.
Note that I don't copy waste space, so it isn't true cloning.
You could try something similar as a disk cloning tool - I use rsync which only copies those parts of a file which have changed.

1 members found this post helpful.

There is an enhanced version of dd called dcfldd and it's much faster than the standard dd command.

Here is a package description of dcfldd

Another tool for cloning disks is called clonezilla. It is a specialty linux distro.

1 members found this post helpful.

I don't quite understand what you mean. As you will have seen, I finally chose 500Mbyte as the appropriate setting for 'bs', rejecting 250K because of its adverse effect on the data transfer rate. Maybe 512K is a bit more logical than 500K but you will note that the latter gave a transfer rate of about 30Mbyte/sec, which was not exceeded by setting bs to 1M or even 2M, suggesting that 30Mbyte/sec is the limit and bs=500K (or 512K, if you prefer) the appropriate setting.

Well, yes. I hoped that that would be apparent from my post. My apologies if it wasn't. I was just rather surprised that this published syntax should fail in the way it did. Luckily, for cloning, it is unnecessary to include the conv=noerror parameter. Any error invalidates a cloning exercise so it does not matter whether or not a failed record is padded out with zeroes to maintain synchronization.

I got my information from http://www.differencebetween.net/tec...0-and-usb-3-0/ which, together with other sources, suggests that the maximum data transfer rate for USB 2.0 is 60Mbyte/sec.

I can remember when differential back-ups were all the rage. It was in the days when hard disks were expensive and a 20Mbyte disk (Yes, I do mean 20Mbyte, not 20Gbyte!) was a big one. We still backed up on magnetic tape at that time. Hard disk capacity was at a premium.

I had to do a differential restore on a couple of occasions. The last full back-up had first to be loaded, followed by all the differential ones in the correct sequence. It took an age. As technology advanced, space saving became less of a concern. Hard disks got steadily bigger and cheaper so, as someone rightly pointed out, "Why bother with differential back ups? Just copy the lot. It's quick, cheap and simple." It really does not matter that the OS itself is included in a disk clone, even if that's not strictly necessary. It probably occupies less than 1Gbyte which, at 30Mbyte/sec, can be transferred in around 30 seconds. My disks are 80Gbyte and cloning now takes little longer than watching the evening news - which I might well do while I am waiting. Things would undoubtedly look different if my disks were of terabyte size but, as I observed earlier, "It's horses for courses."

I mentioned earlier that I have tried EaseUS Todo backup and RedoBackup both of which failed to identify a blank target disk, the former actually crashing the whole computer, forcing a reboot. RedoBackup was not so unkind as to take the computer down with it. I also had a look at Clonezilla which, even its advocates appear to acknowledge, is not the simplest tool to use.

That is interesting. I will certainly follow up the lead but I am beginning to suspect that I am not going to get much faster cloning. Still, with disks of 'only' 80GB, forty five minutes or so to clone them is not unacceptable. Things would, be different if I were talking of 80TB drives but that is not going to happen.

I mentioned Clonezilla in my previous post. I haven't tried it but 'User-friendly' is a term that few reviewers seem willing to apply to it.

Sorry for my confusion between MB & GB (and KB and MB too).

But I stand by the "ignore read errors" is a crazy thing to do on a backup.

If you must clone all data, why not just copy the files, because everything (almost) is a file in Linux.
If you then only copy changed files, hardly any time is taken (after the first copy).
BTW This allows you to ignore web caches and thumbnail caches.

Don't forget that taking differential backups allows you to recover deleted/corrupted data, even if it takes a few weeks to notice that it has gone missing.