LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices


Reply
  Search this Thread
Old 02-04-2016, 03:40 AM   #1
paul2015
Member
 
Registered: Apr 2015
Distribution: CentOS Fedora
Posts: 149

Rep: Reputation: 4
rsync memory usage


hellow everyone,

I backup my webserver's data daily with rsync. after backup it caches memory and when web server is loaded memory is swapping. If i reboot server after rsync backup web server does not swap memory. So it happens because of rsync's memory usage. Can you suggest how to limit rsync memory usage. I do incremental backups with rsync.

thank you.
 
Old 02-04-2016, 04:10 AM   #2
juandresen
LQ Newbie
 
Registered: Dec 2015
Location: Stockholm
Distribution: Gentoo, Debian
Posts: 8

Rep: Reputation: Disabled
Have you checked if your target hard drive has enough space for the backup?
 
Old 02-04-2016, 04:13 AM   #3
paul2015
Member
 
Registered: Apr 2015
Distribution: CentOS Fedora
Posts: 149

Original Poster
Rep: Reputation: 4
yes it has 700BG free backup size is max 11GB, ram on server is 8GB
 
Old 02-04-2016, 04:34 AM   #4
juandresen
LQ Newbie
 
Registered: Dec 2015
Location: Stockholm
Distribution: Gentoo, Debian
Posts: 8

Rep: Reputation: Disabled
Did you also check if your ionodes aren't all used?

$ df -i
 
Old 02-04-2016, 04:34 AM   #5
juandresen
LQ Newbie
 
Registered: Dec 2015
Location: Stockholm
Distribution: Gentoo, Debian
Posts: 8

Rep: Reputation: Disabled
can you post your rsync command?
 
Old 02-04-2016, 04:50 AM   #6
paul2015
Member
 
Registered: Apr 2015
Distribution: CentOS Fedora
Posts: 149

Original Poster
Rep: Reputation: 4
rsync command:

rsync -Aaxv

inodes:
used free
166 60882778
 
Old 02-04-2016, 05:24 AM   #7
juandresen
LQ Newbie
 
Registered: Dec 2015
Location: Stockholm
Distribution: Gentoo, Debian
Posts: 8

Rep: Reputation: Disabled
after using rsync the memory should be released. if indeed the memory does not get released it might point to a memory leak somewhere. i found that other people had the same problem, a possible (but I do not know if recommendable) solution is to drop the cache after rsync

sudo sh -c "sync; echo 3 > /proc/sys/vm/drop_caches"

see

https://www.linuxquestions.org/quest...memory-758345/

http://askubuntu.com/questions/46565...c-memory-usage
 
Old 02-04-2016, 05:25 AM   #8
Guttorm
Senior Member
 
Registered: Dec 2003
Location: Trondheim, Norway
Distribution: Debian and Ubuntu
Posts: 1,295

Rep: Reputation: 335Reputation: 335Reputation: 335Reputation: 335
Hi

rsync uses memory depending on how many files are rsynced. So one way to limit memory usage is to do it in many operations instead. Version 3 and newer use less than older, but both sender and receiver has to be version 3 or newer.

But are you sure it really is a problem? Typically, you configure the server to use more memory for the web server and database. When rsync runs, it needs more. It can then swap out unused memory - which can be unused services, video memory and such things. When rsync is finished, that unused memory stays in the swap until it's needed. If it's not needed, it will just stay there. You now have more free memory, which is typically used for file cache and kernel buffers.
 
Old 02-04-2016, 05:27 AM   #9
paul2015
Member
 
Registered: Apr 2015
Distribution: CentOS Fedora
Posts: 149

Original Poster
Rep: Reputation: 4
yes will try but php files and web server data is also cached in memory. I think it will be safe to drop cache.
 
Old 02-04-2016, 05:33 AM   #10
Guttorm
Senior Member
 
Registered: Dec 2003
Location: Trondheim, Norway
Distribution: Debian and Ubuntu
Posts: 1,295

Rep: Reputation: 335Reputation: 335Reputation: 335Reputation: 335
Why drop the cache? All it does is to drop the file cache, so it will have to be read again later. It will just make things slower.

Is the reason simply that you see swap is uses with for example top? Then I think you are mistaken. It's a good thing that unused memory is put in swap. You now have more memory for file system cache and buffers.
 
Old 02-04-2016, 05:37 AM   #11
paul2015
Member
 
Registered: Apr 2015
Distribution: CentOS Fedora
Posts: 149

Original Poster
Rep: Reputation: 4
yes reason is that and may be it's my mistake or don't understan something but, as I know if apache data will be swapped it will slow down server. so it's memory in swap from rsync not webserver's data?
 
Old 02-04-2016, 05:38 AM   #12
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 15,996

Rep: Reputation: 2219Reputation: 2219Reputation: 2219Reputation: 2219Reputation: 2219Reputation: 2219Reputation: 2219Reputation: 2219Reputation: 2219Reputation: 2219Reputation: 2219
Listen to @Guttorm - if you don't know what you are looking at, how can you make a proper decision ?.
 
Old 02-04-2016, 05:40 AM   #13
juandresen
LQ Newbie
 
Registered: Dec 2015
Location: Stockholm
Distribution: Gentoo, Debian
Posts: 8

Rep: Reputation: Disabled
If the data in the memory will be used only every 24 hours, there is really no need to keep it there. Moreover, if other programs start to swap, it could actually be a memory leak, so that memory would not be used again.
 
Old 02-04-2016, 10:08 AM   #14
paul2015
Member
 
Registered: Apr 2015
Distribution: CentOS Fedora
Posts: 149

Original Poster
Rep: Reputation: 4
thanks guys, my mistake now it's clear little bit. I have also read this now I have clear understanding about that. and it has solved my problem.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Shell script for CPU usage, memory usage, disk partition space and service status reetesh.amity Linux - Server 6 10-12-2015 08:51 PM
rsync memory usage over ssh tajamari Linux - Software 9 09-20-2007 06:52 AM
X hangs after prolonged usage ... memory usage goes upto 195MB duryodhan Slackware 18 05-10-2007 07:16 PM
getting realtime info on memory usage-cpu and harddrive usage steering Linux - Newbie 5 03-03-2005 09:43 PM
how to determine cpu usage, memory usage, I/O usage by a particular user logged on li rags2k Programming 4 08-21-2004 05:45 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Server

All times are GMT -5. The time now is 01:36 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration