LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Hardware
User Name
Password
Linux - Hardware This forum is for Hardware issues.
Having trouble installing a piece of hardware? Want to know if that peripheral is compatible with Linux?

Notices


Reply
  Search this Thread
Old 12-01-2005, 02:39 AM   #1
Diagmato
Member
 
Registered: Jul 2005
Location: Cardiff
Distribution: Suse 10
Posts: 59

Rep: Reputation: 15
Backing up took 3 days


I have recently started using suse 10. Before that I was on fedora core 4.

Every few weeks, i turn on the external drive, and run:

cp -fvr /home/james/.* /media/backups/home
cp -fvr /mnt/win_c2/.* /media/backups/home

On fedora core, and way back on mandriva 10.2, this only took a few hours - possibly 5 at most. On suse, its taken 3 days, and is still going.

The drive is USB2.0 - a possibility is that its only running at usb1's speed? If so how can i check this?

Anything else it could be? There isnt really that much more files to be backed up since fedora core 4, so its not as if theres thousands of large files (since the last backup).
 
Old 12-01-2005, 05:48 PM   #2
tredegar
LQ 5k Club
 
Registered: May 2003
Location: London, UK
Distribution: Fedora38
Posts: 6,147

Rep: Reputation: 435Reputation: 435Reputation: 435Reputation: 435Reputation: 435
Backups of my home directory of several GB take me less than 10 minutes with an iee1394 external drive. It is much slower (2-3 Hrs) with USB.
Quote:
cp -fvr /home/james/.* /media/backups/home
cp -fvr /mnt/win_c2/.* /media/backups/home
Your cp command is strange. Why have you chosen the arguments fvr and why have you chosen /home/james/. * or /mnt/win_c2/.*as your sources?

What exactly are you trying to achieve?
Please see man cp
HTH
 
Old 12-01-2005, 06:52 PM   #3
Diagmato
Member
 
Registered: Jul 2005
Location: Cardiff
Distribution: Suse 10
Posts: 59

Original Poster
Rep: Reputation: 15
v was for verbose mode, so i could see it was actually going through the correct files. r for recursive, so it would copy all subdirectories and files. f to force the file to re-copy if the file cannot be opened?

the .* was to copy all files including the /home hidden folders (e.g, /home/james/.kde).

The general idea is to backup the entire contents of both the home directory, and the documents directory (was used as a shared partition back when i dual booted with windows - kept it like that ever since).

I thought the difference in transfer rate between ieee1394 and usb2.0 (480 mbps) was just 80mbps? (ieee1394 being 400mbps). I can only think that it transferred at usb 1's speed, but for the sake of a cable, seems worth using ieee1394.

Thanks for the help - would like to know where im going wrong with that command. Couldnt see anything wrong as far as the man pages stated.

Last edited by Diagmato; 12-01-2005 at 06:57 PM.
 
Old 12-02-2005, 01:40 AM   #4
tredegar
LQ 5k Club
 
Registered: May 2003
Location: London, UK
Distribution: Fedora38
Posts: 6,147

Rep: Reputation: 435Reputation: 435Reputation: 435Reputation: 435Reputation: 435
Quote:
the .* was to copy all files including the /home hidden folders (e.g, /home/james/.kde).
Couldnt see anything wrong as far as the man pages stated.
So, for example, what does the output of ls /home/james/.*
and ls -R /home/james/.* (the recursive option for ls)
Look like?

Is this really what you want? Watch carefully how the shell handles this last command.

And why do you want to "force the file to re-copy if the destination file cannot be opened?"
 
Old 12-02-2005, 11:40 AM   #5
Diagmato
Member
 
Registered: Jul 2005
Location: Cardiff
Distribution: Suse 10
Posts: 59

Original Poster
Rep: Reputation: 15
ls /home/james/.* - listed alot of folders, and a couple files in each folder.
ls /home/james/* seemed to do exactly the same.

ls -R /home/james/.* - listed thousands of files and folders. Although it seemed as if it went through more than once? Was difficult to see with the speed it went through them.

ls -R /home/james/* - listed what seems like hardly anything - less than the first two commands. (edit - it didnt list anything in a hidden folder - anything which name started with a '.')

I used the 'f' flag as someone told me it re-copies the file if it somehow got corrupt. didnt want to risk the backup getting corrupt at all.

All i want is to backup every file/folder/subdirectory - everything within the /home and /mnt/win_c2 directories without fault. I used to do this with konqueror or nautilus, but both seemed to either miss out some files, or the amount of space taken up on the external drive was gigabytes smaller than that on the pc (same filesystem on both drives - ext3).

Last edited by Diagmato; 12-02-2005 at 11:42 AM.
 
Old 12-02-2005, 04:55 PM   #6
Electro
LQ Guru
 
Registered: Jan 2002
Posts: 6,042

Rep: Reputation: Disabled
You can use cp -av. This make sure everything will be copied.

Quote:
I thought the difference in transfer rate between ieee1394 and usb2.0 (480 mbps) was just 80mbps? (ieee1394 being 400mbps).
It is not the speed. It is the latency. USB has higher latency than Firewire (IEEE-1394). Firewire was design for multimedia transfer. USB is design for low IO devices such as printers and scanners.

On my computer that uses USB version 1.1 it goes at the rate of 800 KB per second. It takes about a few days to copy about 80 GB of data. If I get USB version 2.0, it would go about 25 MB per second, but only if I use XFS and the drive can handle it.
 
Old 12-02-2005, 05:10 PM   #7
tredegar
LQ 5k Club
 
Registered: May 2003
Location: London, UK
Distribution: Fedora38
Posts: 6,147

Rep: Reputation: 435Reputation: 435Reputation: 435Reputation: 435Reputation: 435
Quote:
ls /home/james/.* - listed alot of folders, and a couple files in each folder.
ls /home/james/* seemed to do exactly the same.
Sigh. There's a difference. I fear you are not paying sufficient attention here.

"seemed to do exactly the same" doesn't work in real life, or linux:

ls /home/james/.* listed all the files that were in the directory /home/james/ and began with .anything. (Please notice the red dot there). It did not list any files that did not begin with a dot. (So thinking back to your "backup" command, you did not backup any files that did not begin with a dot. This is probably NOT what you intended)

ls /home/james/* listed all the files in /home/james except those whose names began with .
If you had wanted ALL the files, whether they began with a dot or not you'd use the -a argument to ls, like this:

ls -a /home/james/*

You say it "was difficult to see with the speed it went through them." (Hey, linux is fast, this is nice). If the output goes too fast, you can either use the scroll bar in your window to review the output, or re-direct the output to a file and then read it with an editor (and I am assuming you are using a graphical desktop), eg like this:

ls /home/james/.* > /home/james/myfile

kwrite /home/james/myfile


And then you should compare this to the output from

ls /home/james/.* > /home/james/my_ls-a_file

kwrite /home/james/my_ls-a_file


See the difference?

I am hammering this point because if you do not understand the difference between .* and * you will soon be lost with linux.

Please do your homework (man bash is worth reading several times, please start now.)

OK. We have had a diversion. Getting back to why your "backup" took so long:

You fed cp the r flag to signify recursion. You also fed cp a path of .* and DOT STAR matches anything that starts with a DOT. This includes the special file in all directories (except root) .. which refers to the directory above the current one (By now you have read man bash haven't you?)

If you ask cp to recurse through the directory tree, including .. , it will end up at / and then follow directories it hasn't scanned yet. One of these directories will be /mnt which is no doubt where your external disk is mounted so it will try copying the external disk to the external disk to the external disk.....
Well, you get the picture (a recursive nightmare). The process will probably end with "Disk full", if you wait long enough.

In short, your choice of "backup" command is wrong. I am trying to show you why it is wrong.

My email just went "Ping" and I have just read Electro's post. With respect Electro, I'm trying to point Digamato in the right direction......and tell him why, so he can do it for himself the next time.

HTH
 
Old 12-02-2005, 06:17 PM   #8
Diagmato
Member
 
Registered: Jul 2005
Location: Cardiff
Distribution: Suse 10
Posts: 59

Original Poster
Rep: Reputation: 15
I understand about the .* meaning anything beginning with a dot. However, it backup all the files in the home directory, including anything which didnt start with a dot. It also didnt go up a directory, which is one of the points which confused me loads, because all other commands ("cd some*" instead of "cd something" to avoid typing out the full name) ive tried would do otherwise.

To back this up, comparing the directory and the backed up directory are exactly the same size, and contain the same number of files and folders. I dont know why its done this, like as you said, it should have gone up a directory, and only done anything beginning with a dot. I could also screenshot the directory properties if needs be.

I did the ls commands again, outputting to a file, and looked over them. The .* does as you say - it only includes files/folders that start with a dot. I just dont get why the cp command included every file/folder. Also, ls -R /home/james/.* included files within the specified directory, jumped up a directory, then went through the home directory again. Baring that in mind, the /home/james/ directory, and the backup directory are the same size :s.

I have been reading the man file, but "if an existing destination file cannot be opened, remove it and try again" sounded to me like "if there is something wrong with the file, retry". That probably sounds rather misinterpreted, but i havent used the console very much.

Sorry for all the confusion, and thanks for the help.
 
Old 12-03-2005, 11:47 AM   #9
tredegar
LQ 5k Club
 
Registered: May 2003
Location: London, UK
Distribution: Fedora38
Posts: 6,147

Rep: Reputation: 435Reputation: 435Reputation: 435Reputation: 435Reputation: 435
OK, you seem to be getting there!
Generally, I use tar for backing up, as that is just what it was designed to do.
The following will backup EVERYTHING in /home/james for you, just perfectly:

Code:
md /media/james-home-backup-todaysDate
chown james /media/james-home-backup-todaysDate
cd /home/james
tar cf - . | (cd /media/james-home-backup-todaysDate && tar xBfp -)
All files and directories (hidden or not) will be copied, with ownership, permissions, date & timestamps etc. preserved for you.

Or, you can try the cp command as Electro suggested.

When backing up my home, I generally make sure I am not logged in as myself, as being logged in & running KDE alters some of the files. So I login as another user, use su to become root, and then run the commands as in the box above.

Try the above, and time it. As I said, with ieee1394, it is blindingly fast, but slower with USB. I think your "slow backup" was due to using the wrong command, not a fault with USB, but I stand to be corrected.
Best wishes.
 
Old 12-04-2005, 07:21 AM   #10
Diagmato
Member
 
Registered: Jul 2005
Location: Cardiff
Distribution: Suse 10
Posts: 59

Original Poster
Rep: Reputation: 15
Thanks! I ran that command, and it does what I want.

The external case caddy seems "worn out" - turned it on, and it sounded alot slower than usual, and there were strange static noises every now and then which stopped after abour 5 mins. Using files on the drive doesnt seem any different though, so I hope its just the external case thats breaking. I guess its not meant to be on for 3 days straight.

Ill get an ieee1394 case caddy, and will use your command for backing up - thanks for the help!
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
The good old days! twirl General 5 09-23-2005 10:48 PM
Crontab every 15 days, isn't possible? deqmacrom Linux - General 3 04-07-2005 02:31 AM
having one of THOSE DAYS ! qwijibow General 3 11-10-2004 04:13 AM
once in how often? twice in two days!? andzerger Linux - Security 1 02-20-2004 05:45 AM
Seven days to install an OS! raid517 Linux - Networking 4 06-28-2003 07:37 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Hardware

All times are GMT -5. The time now is 06:38 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration