LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices


Reply
  Search this Thread
Old 04-28-2008, 07:40 AM   #1
dhanju
Member
 
Registered: Aug 2005
Location: bangalore
Posts: 47

Rep: Reputation: 18
Rsync utilizing CPU to 100% !


Hi,

I am using Rsync cmd for copying files between sorce and target box daily. It have to check incremental chunks out of 20 GB and copy the changed chunks from source and paste it to target. It takes approx. 10 - 20 Mins. During this period CPU in source and target is reaching up to 100% utilization.
Kindly, suggest me how to control CPU utilization or what else can be done.
 
Old 04-28-2008, 07:48 AM   #2
jbilla
LQ Newbie
 
Registered: Apr 2008
Posts: 19

Rep: Reputation: 0
Thats because while copying from rsync command it tries to retrieve the list of files that you are copying and because you have too many files so in memory its utilize too much ram as well as resources to copy them to new place.. You will have to distribute the task among small task..

You will have to divide your operation for rsync in bash script like this way..
Like for example this is your script
rsync /source root@1.1.1.1:/source_backup/

THEN
Create a bash script and apply it in this way

#!/bin/bash
rsync /source/[0-9]* root@1.1.1.1:/source_backup/
rsync /source/[a-h]* root@1.1.1.1:/source_backup/
rsync /source/[i-p]* root@1.1.1.1:/source_backup/
rsync /source/[q-z]* root@1.1.1.1:/source_backup/

Last edited by jbilla; 04-28-2008 at 07:52 AM.
 
Old 04-28-2008, 11:00 AM   #3
tredegar
LQ 5k Club
 
Registered: May 2003
Location: London, UK
Distribution: Fedora38
Posts: 6,147

Rep: Reputation: 435Reputation: 435Reputation: 435Reputation: 435Reputation: 435
Or, man nice
 
Old 04-30-2008, 05:48 AM   #4
tajamari
Member
 
Registered: Jul 2007
Distribution: Red Hat CentOS Ubuntu FreeBSD OpenSuSe
Posts: 252

Rep: Reputation: 32
Quote:
Originally Posted by dhanju View Post
Hi,

I am using Rsync cmd for copying files between sorce and target box daily. It have to check incremental chunks out of 20 GB and copy the changed chunks from source and paste it to target. It takes approx. 10 - 20 Mins. During this period CPU in source and target is reaching up to 100% utilization.
Kindly, suggest me how to control CPU utilization or what else can be done.
that's the behavior of rsync. you can do chunk rsync to avoid such usage.
 
Old 04-30-2008, 09:41 AM   #5
jbilla
LQ Newbie
 
Registered: Apr 2008
Posts: 19

Rep: Reputation: 0
Quote:
Originally Posted by tajamari View Post
that's the behavior of rsync. you can do chunk rsync to avoid such usage.
What you mean by "chunk rsync" here? Could you please elaborate?
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
100% cpu use 100% of the time serafean Linux - Newbie 4 05-04-2006 03:36 PM
cpu ......100%...... mohama Linux - General 23 01-20-2006 10:39 AM
100 % Cpu Volcano Linux - Newbie 27 10-05-2005 10:58 PM
rsync reduce cpu usage dtra Linux - Software 0 05-25-2005 08:04 PM
CPU on 100% very often Satriani Linux - General 25 12-23-2003 03:41 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Server

All times are GMT -5. The time now is 02:35 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration