Linux - ServerThis forum is for the discussion of Linux Software used in a server related context.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
I'm using rsync on my fileserver to backup the data. The problem is that rsync doesn't seem to release the memory it uses. After booting ~20 MB of memory is in use. This jumps up to 270 MB after running rsync. I know rsync uses quite a bit of memory when it's running, but surely it should be released upon completion.
Here's the situation after booting the system (free -m):
Linux uses all the RAM it can find for active progs and caches. If a program exits, the RAM remains cached until another process needs it, at which pt the kernel will flush the old/unwanted stuff and re-use the RAM for the new proc.
The kernel handles all this transparently.
IOW, ignore it, it's not an issue.
I knew someone would say that That's why I posted the output of the free command in the first place.
I know about Linux caching. However, if you take a look at the output of the free command above, you'll notice that 480 MB of total memory is in use and only 156 MB cached + 53 MB buffered (i.e. 209 MB), so I'm still missing the mentioned 270 MB (well, 271 MB actually, but that's a rounding error). It _is_ an issue because after a couple of rsyncs almost all the memory is used (disappeared, not cached+buffered) and the system becomes very unresponsive.
Wow . . . well, if you really think that rsync is leaking or failing to free memory, then I would suggest building a fresh version from source or downloading another binary and seeing if it still happens. I know that rsync does not do this to me.
If it still happens, the problem may not be with rsync.
Nope, it's not Slack 13, it's 11 (with all the available upgrades installed, though). So far I've seen no reason to upgrade it since 11 really does everything I want it to do. rsync is version 2.6.9 on this system. I'll try to build a newer one to see if that works better.
I knew someone would say that That's why I posted the output of the free command in the first place.
I know about Linux caching. However, if you take a look at the output of the free command above, you'll notice that 480 MB of total memory is in use and only 156 MB cached + 53 MB buffered (i.e. 209 MB), so I'm still missing the mentioned 270 MB (well, 271 MB actually, but that's a rounding error). It _is_ an issue because after a couple of rsyncs almost all the memory is used (disappeared, not cached+buffered) and the system becomes very unresponsive.
I've seen the same think after rsyncing a largish directory tree. From the bit of research I did, it seems to be attributable to dentries /inode buffers. You can flush those buffers and reclaim the memory by doing:
But I don't think it's really necessary. Linux will eventually reclaim the memory when needed.
A note about rsync: it's recommended to have the same rsync version on both ends of the transfer for best performance and to avoid possible issues. And v. 3 brings some important changes -- have a look through the release notes.
Last edited by sleddog; 10-07-2009 at 08:10 PM.
Reason: Added notes about rsync version
I've seen the same think after rsyncing a largish directory tree. From the bit of research I did, it seems to be attributable to dentries /inode buffers. You can flush those buffers and reclaim the memory by doing:
But I don't think it's really necessary. Linux will eventually reclaim the memory when needed.
A note about rsync: it's recommended to have the same rsync version on both ends of the transfer for best performance and to avoid possible issues. And v. 3 brings some important changes -- have a look through the release notes.
I'm running RHEL5.4 and am using rsync for backup onto a 2nd disk (something like poor man's RAID 1). This worked for me. Thanks!
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.