LinuxQuestions.org
Support LQ: Use code LQ3 and save $3 on Domain Registration
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices

Reply
 
Search this Thread
Old 08-20-2006, 05:38 PM   #1
Sammael
Member
 
Registered: Jun 2006
Location: Slovakia, currently working in Liverpool UK
Distribution: Gentoo 2006.0
Posts: 97

Rep: Reputation: 15
copying over lan eats CPU


whenever i copy something over lan, my cpu usage goes nearly to 100%, even if i am copying to another drive. is this ok? i mean, it's only copying and it really slows down my system... can this behaviour be changed somehow?

if it matter's the other drive is an external usb2 toshiba 320gb drive. first i observed this behaviour on my primary disk, but i thought it was because i was copying data to system drive. so i bought an external one, to take off some load from my system disk.

if this behaviour is normal, can someone please explain to me what exactly eats that cpu while copying?

thanks
 
Old 08-20-2006, 06:26 PM   #2
MensaWater
Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 6,009
Blog Entries: 5

Rep: Reputation: 787Reputation: 787Reputation: 787Reputation: 787Reputation: 787Reputation: 787Reputation: 787
You need to let us know exactly what you're doing when you're copying. Something like "cp /directory1/file1 /directory2" that only copies a single file shouldn't be CPU intensive. However doing something like:

for FILE in `find / `
do cp $FILE /mnt/usbdrivemountname &
done

Would spawn a new process for each file and would hammer the CPU because each of the copies would simultaneously be trying to open the binar (cp) and any of the unshared libraries. Often I see people new to scripting doing something like the above. The issue isn't so much what you're telling it to do but how you're telling it to do it. In the above example it being told to background the cp of one file then immediately start another copy whether the first one completed or not and so on until it reaches some limit (memory, cpu, open file handles). That example could be mollified simply by taking out the ampersand (&) as then it wouldn't start the copy of file 2 until file 1 was done and file 3 would wait for file 2 etc...

Copying files by its nature should be mainly I/O (disk read/disk write) intensive rather than CPU or memory although of course both are affected. Often seeing something like the CPU being hammered means either you've told it to do the operation in an inefficient manner or there is a problem with your I/O devices that is requiring brute CPU power to overcome.

If you give us an example of exactly what you're doing we may be able to help debug it for you.

Also - are you looking at CPU utilization before you start? If you're running at 90% utlization already it won't take much to put you 100%.
 
Old 08-20-2006, 07:25 PM   #3
Sammael
Member
 
Registered: Jun 2006
Location: Slovakia, currently working in Liverpool UK
Distribution: Gentoo 2006.0
Posts: 97

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by jlightner
Also - are you looking at CPU utilization before you start? If you're running at 90% utlization already it won't take much to put you 100%.
yes, my cpu utilization before copying is between 5-10 %

Quote:
Originally Posted by jlightner
You need to let us know exactly what you're doing when you're copying.
i have opened konqueror, two tabs, one of which points to smb://xxx.xxx.xxx.xxx/Movies, the other one to /media/usbdisk/download. i took all directories inside "samba tab" and dragged them to "local tab", then KDE Copy Dialog appears and instantly my computer is nearly unusable. i would understand it if i'd have 500 MHz processor, but mine config is 2800 MHz Intel Celeron with 768 MB Ram.

Quote:
Originally Posted by jlightner
Copying files by its nature should be mainly I/O (disk read/disk write) intensive rather than CPU or memory although of course both are affected. Often seeing something like the CPU being hammered means either you've told it to do the operation in an inefficient manner or there is a problem with your I/O devices that is requiring brute CPU power to overcome.
i also have that feeling something is wrong, please let me know if any log files would be helpful to post.

Quote:
Originally Posted by jlightner
If you give us an example of exactly what you're doing we may be able to help debug it for you.
that would be most appreciated
 
Old 08-20-2006, 09:00 PM   #4
jerp
LQ Newbie
 
Registered: Aug 2006
Posts: 5

Rep: Reputation: 0
It's your client or server SMB configuration. I've seen it, but can't remember what I did to fix it. I use NFS rather than SMB. I don't see this problem with NFS.
 
Old 08-21-2006, 05:05 AM   #5
Sammael
Member
 
Registered: Jun 2006
Location: Slovakia, currently working in Liverpool UK
Distribution: Gentoo 2006.0
Posts: 97

Original Poster
Rep: Reputation: 15
well, i need to use smb, as i need to connect to windoze box
 
Old 08-21-2006, 07:14 AM   #6
dsids
Member
 
Registered: Mar 2006
Distribution: FC4
Posts: 184

Rep: Reputation: 31
Quote:
Originally Posted by jlightner
You need to let us know exactly what you're doing when you're copying. Something like "cp /directory1/file1 /directory2" that only copies a single file shouldn't be CPU intensive. However doing something like:

for FILE in `find / `
do cp $FILE /mnt/usbdrivemountname &
done

Would spawn a new process for each file and would hammer the CPU because each of the copies would simultaneously be trying to open the binar (cp) and any of the unshared libraries. Often I see people new to scripting doing something like the above.
Hi,
I was tempted to ask a question after reading your article.
I wanted to know:
if I do on (suppose) pts/5:

$cp file1 file2 file3 /path/to/dir

Would this command copy all the files by spawing just one process or would it spawn just one process.
From what I understood in your article if I use the above command in a shell script, then a new process would be spawned for each file:

#!/bin/sh

cp file1 file2 file3 /path/to/dir



Thanks
Danish
 
Old 08-21-2006, 02:47 PM   #7
MensaWater
Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 6,009
Blog Entries: 5

Rep: Reputation: 787Reputation: 787Reputation: 787Reputation: 787Reputation: 787Reputation: 787Reputation: 787
Quote:
Originally Posted by dsids
Hi,
I was tempted to ask a question after reading your article.
I wanted to know:
if I do on (suppose) pts/5:

$cp file1 file2 file3 /path/to/dir

Would this command copy all the files by spawing just one process or would it spawn just one process.
From what I understood in your article if I use the above command in a shell script, then a new process would be spawned for each file:

#!/bin/sh

cp file1 file2 file3 /path/to/dir



Thanks
Danish
It should just spawn one process. The binary, cp, would be executed and it would handle the routines for copying each of the source files to the target directory. In my example it would have opened a new copy of cp for each file to be copied.

Typically I don't use the syntax you show because in the old days it wasn't an option. cp file1 file2 file3 /path/to/dir would likely have copied file1 OVER file2 and ignored the rest of the command line. For Linux cp it is an option but its hard to teach old dogs new tricks.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Remote administration eats all my CPU :( tomazN Linux - Software 7 06-22-2005 09:38 AM
FireFox eats 98% of my CPU athens101 Linux - Software 5 03-19-2005 04:35 PM
Cpu usage while copying files from DVD TheCondor389 Suse/Novell 2 01-07-2005 12:16 PM
samba? copying files via lan from linux to winNT? egypt Linux - Networking 3 09-28-2004 07:30 AM
kio_thumbnail eats user CPU gavinbeatty Mandriva 0 06-15-2004 11:29 AM


All times are GMT -5. The time now is 07:02 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration