LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 10-01-2013, 07:40 AM   #1
sysbox
Member
 
Registered: Jul 2005
Posts: 117

Rep: Reputation: 15
Limit Disk Cache Size


My machine has 24 GB of memory and runs CentOS 6.4 64-bit. I'm having problems using /dev/shm. I think this is because the disk cache is filling up (see previous thread). I don't know how to fix it, but if I limit the disk cache size, then that's might work.

Does anyone know how to limit the size of the disk cache?
 
Old 10-01-2013, 08:45 AM   #2
camorri
LQ Guru
 
Registered: Nov 2002
Location: Somewhere inside 9.9 million sq. km. Canada
Distribution: Slackware 14.1, 14.2
Posts: 5,143

Rep: Reputation: 503Reputation: 503Reputation: 503Reputation: 503Reputation: 503Reputation: 503
You can change the size of /dev/shm on the fly with a command like this :

Quote:
mount -o remount,size=40G /dev/shm
Make the size as you see fit. The default is half of your ram. If you have a swap file, and /dev/shm fills up, the kernel will swap some of it out to swap.

You can display what is in /dev/shm with a 'df' command.
 
Old 10-01-2013, 09:09 AM   #3
sysbox
Member
 
Registered: Jul 2005
Posts: 117

Original Poster
Rep: Reputation: 15
I don't want to change the size of /dev/shm. I want to change the size of the disk cache. Why? Because the disk cache is getting too large and that is taking space away from /dev/shm. That prevents me from being able to use /dev/shm (and it prevents other apps from using /dev/shm, and those apps are crashing).

How can I decrease the size of the disk cache?

Or, how can I prevent the disk cache taking space away from /dev/shm? I know that the disk cache is not supposed to take space away from other applications. Maybe this is true. But it is certainly taking space away from /dev/shm. So the disk cache is effectively taking memory space away from other applications. Thank you.
 
Old 10-01-2013, 10:16 AM   #4
camorri
LQ Guru
 
Registered: Nov 2002
Location: Somewhere inside 9.9 million sq. km. Canada
Distribution: Slackware 14.1, 14.2
Posts: 5,143

Rep: Reputation: 503Reputation: 503Reputation: 503Reputation: 503Reputation: 503Reputation: 503
Quote:
How can I decrease the size of the disk cache?
Are you asking about memory cache ( ram ) or hard drive cache memory (swap)?
 
Old 10-01-2013, 10:36 AM   #5
camorri
LQ Guru
 
Registered: Nov 2002
Location: Somewhere inside 9.9 million sq. km. Canada
Distribution: Slackware 14.1, 14.2
Posts: 5,143

Rep: Reputation: 503Reputation: 503Reputation: 503Reputation: 503Reputation: 503Reputation: 503
If its the buffers, and cache in memory, you may find this link useful.

-->http://www.unixmen.com/how-to-clear-...linux-servers/
 
Old 10-01-2013, 10:45 AM   #6
sysbox
Member
 
Registered: Jul 2005
Posts: 117

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by camorri View Post
Are you asking about memory cache ( ram ) or hard drive cache memory (swap)?
I'm not sure (newbie). I'm asking about the cache which is reported in /proc/meminfo on the fourth line which begins with 'Cached:'. As this number grows, the available space I can use in /dev/shm decreases, and this causes my app (which uses /dev/shm for tmp files) to fail.
 
Old 10-01-2013, 10:49 AM   #7
sysbox
Member
 
Registered: Jul 2005
Posts: 117

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by camorri View Post
If its the buffers, and cache in memory, you may find this link useful.

-->http://www.unixmen.com/how-to-clear-...linux-servers/
Yes, I think that will work for me. I will try it later. However, can you think of a reason why this cache isn't automatically free'ed as I copy files into /dev/shm? For 3 years, on this machine, this worked fine. But for some reason, about a month ago, the memory cache seems to be preventing me from using /dev/shm, and this is causing all sorts of problems. Could I have inadvertently changed something which altered this behavior?
 
Old 10-01-2013, 11:04 AM   #8
camorri
LQ Guru
 
Registered: Nov 2002
Location: Somewhere inside 9.9 million sq. km. Canada
Distribution: Slackware 14.1, 14.2
Posts: 5,143

Rep: Reputation: 503Reputation: 503Reputation: 503Reputation: 503Reputation: 503Reputation: 503
Quote:
I'm asking about the cache which is reported in /proc/meminfo
That is what is in ram, not the hard drive. The sync command as indicated in post #5 can reduce the number of buffers and cache. BTW, you can see the buffers and cache dynamically with the top command.

There are parameters in the kernel config that have an affect on buffers and cache.

Quote:
Could I have inadvertently changed something which altered this behavior?
I believe a kernel upgrade could have had some effect on this. That would take some investigating.

I had a problem recently with /dev/shm in memory. I had problems compiling an application. The work around was to mount temp file space on a real hard drive partition, with adequate space. Once done, the app compiled. This is not an ideal solution for the long term though, since disk access is much slower than memory access.
 
Old 10-01-2013, 11:23 AM   #9
sysbox
Member
 
Registered: Jul 2005
Posts: 117

Original Poster
Rep: Reputation: 15
Okay, then forget disk cache. I think my issue is with the memory cache. The problem seems to be that when the memory cache gets close to the total size of physical memory, I can no longer use /dev/shm.

I did not upgrade the kernel recently, so that could not have caused this problem.

This problem started about a month ago about the time I configured a software RAID with mdadm. Is it possible I changed something that caused the memory cache to behavior differently? I looked at every file in the
/etc directory tree that changed during the past month. No files changed which had anything to do with the disk (other than possibly the mdadm config file).

Where would be a good place to look for answers to this? I really need to be able to use /dev/shm.

Last edited by sysbox; 10-01-2013 at 11:30 AM.
 
Old 10-01-2013, 12:22 PM   #10
rknichols
Senior Member
 
Registered: Aug 2009
Distribution: CentOS
Posts: 3,396

Rep: Reputation: 1485Reputation: 1485Reputation: 1485Reputation: 1485Reputation: 1485Reputation: 1485Reputation: 1485Reputation: 1485Reputation: 1485Reputation: 1485
You might try reducing swappiness to increase the kernel's preference for throwing away cache pages when there are demands for memory.
 
Old 10-01-2013, 12:23 PM   #11
camorri
LQ Guru
 
Registered: Nov 2002
Location: Somewhere inside 9.9 million sq. km. Canada
Distribution: Slackware 14.1, 14.2
Posts: 5,143

Rep: Reputation: 503Reputation: 503Reputation: 503Reputation: 503Reputation: 503Reputation: 503
Quote:
This problem started about a month ago about the time I configured a software RAID with mdadm. Is it possible I changed something that caused the memory cache to behavior differently?
This is beyond my knowledge; unfortunately. However, if the problems started after configuring RAID, it looks like the best bet at this point.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Can I set maximun disk cache size in linux? codec Linux - General 4 12-07-2010 02:25 AM
Postfix - cache? logging? taking up all disk space - how to limit/turn off blast8180 Linux - Software 1 02-22-2010 02:55 PM
assign disk quota or size limit to a specific directory or folder on fedora abhijit_mohanta Linux - General 2 08-10-2009 03:04 AM
Are Quota Limit and Disk Size consumed different? Nakano Linux - Newbie 1 07-19-2009 03:08 AM
how to troubleshoot disk size limit? lxa2xjj Linux - Hardware 2 08-07-2008 01:34 PM


All times are GMT -5. The time now is 10:00 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration