How to disable bdflush on a file system?

Quartz · 01-14-2004, 01:51 PM

Hi,

I store pictures for a survey application (25 images/sec and 4 cameras).
I put a lot of pictures in a single file and files are open in write-only mode.
Those pictures are newer read during recording.

The cache buffer keep a copy of a lot of buffers. With vmstat, buff and cache are going bigger and bigger. After some time, there are "swapping"!
The machine have 1024 Ko RAM.

Even with open(...,O_sync) or fdatasync(...) the problem is the same.

Is it possible to discable cache disk for a file system?
Is it possible to change bdflush parameters in order to have no more cache disk ?

I can send in private a test program (500 lines in C multi-threaded).

Thanks,

Yves Crespin

Code:

$uname -r
2.4.21-0.13mdkenterprise
$cat /proc/sys/vm/bdflush
30      500     0       0       500     3000    60      20      0
#/sbin/update -d
bdflush version 1.4
0:    30 Max fraction of LRU list to examine for dirty blocks
1:   500 Max number of dirty blocks to write each time bdflush activated
2:     0 Num of clean buffers to be loaded onto free list by refill_freelist
3:     0 Dirty block threshold for activating bdflush in refill_freelist
4:   500 Percentage of cache to scan for free clusters
5:  3000 Time for data buffers to age before flushing
6:    60 Time for non-data (dir, bitmap, etc) buffers to age before flushing
7:    20 Time buffer cache load average constant
8:     0 LAV ratio (used to determine threshold for buffer fratricide).

Code:

 r  b   swpd   free   buff  cache   si   so    bi    bo   in    cs us sy id wa
20  6      0 588048  38240 221908    0    0     0  2287  432   405  7 35 58  0
28  0      0 537968  38436 270560    0    0     0  5096  750   465 14 85  1  0
21  7      0 488412  38720 318616    0    0     0  5112  750   514 14 86  0  0
16  8      0 437864  38908 367716    0    0     0  5150  758   481 14 85  1  0
12 13      0 388020  39068 416180    0    0     0  5070  745   466 15 82  3  0
21  6      0 337952  39236 464864    0    0     0  5097  757   460 13 85  2  0
17 11      0 288220  39380 513220    0    0     0  5078  783   516 13 86  2  0
 6 19      0 237520  39608 562448    0    0     0  5178  758   595 14 86  0  0
18  9    876  11956  21956 800768    0    0     0  5027  748   532 11 89  0  0
21  7    876  11704  21532 802576    0   22     0  5037  744   453 12 87  1  0
23  1    876  11516  21448 802836    0   38     0  5029  744   475 10 89  0  0
14 14    876  12304  16632 807300    0   13     0  5018  744   480  9 91  0  0
20  4   1892  12092  16916 809320    0    0     0  5040  810   424  8 91  1  0

SciYro · 01-14-2004, 06:28 PM

not sure what your asking? so how about you start with what file system it is (there are 4 major linux file systems and at least a dozen others that linux can suport in some way, so its a good idea to know what fs you watn to do this with)

jailbait · 01-14-2004, 07:09 PM

I would describe your problem this way:

You are writing a lot of buffers to disk. By using sync you are forcing the cache to write dirty buffers to disk fast enough so that there are very few, if any, present in the cache. However once a dirty buffer is written to disk it is kept in the cache as a clean buffer. Your memory is filling up with clean buffers which you know you will never need to satisfy a read request.

I think that you can change /proc/sys/vm/bdflush with the following command:
echo "30 500 0 0 500 3000 60 20 0" > /proc/sys/vm/bdflush
which I think will set it to the same values as your post shows.

So I would experiment with changing /proc/sys/vm/bdflush. The first thing that I would experiment with is:
2: 0 Num of clean buffers to be loaded onto free list by refill_freelist
and see if you can remove large numbers of clean buffers from the cache. Perhaps:
echo "30 500 500 0 500 3000 60 20 0" > /proc/sys/vm/bdflush
to see if every time you write dirty buffers then you immediately add the same number of clean buffers to the free list.

Of course, any such experiments should be done in a testing environment since you will probably knock Linux down many times before you find out whether this is a workable solution. This experiment may also corrupt your filesystem if it causes Linux to not write dirty buffers that it should write, so run your experiments against a partition that you can afford to reformat after a failed experiment.

___________________________________
Be prepared. Create a LifeBoat CD.
http://users.rcn.com/srstites/LifeBo...home.page.html

Steve Stites

Quartz · 01-15-2004, 02:50 AM

bdflush don't work with VFS (Virtual Files System). I'll have the same problem with all FS.

I have test Ext2 and Ext3.

Yves

Quartz · 01-16-2004, 02:15 AM

I have change a lot parameters in bdflush without any success : the cache became bigger and bigger!
Is it possible to disable bdflush on only a file system?

Thanks,

Yves

jailbait · 01-16-2004, 10:01 AM

"Is it possible to disable bdflush on only a file system?"

I don't think so. I spent a lot of time looking with Google before I posted my first answer. I could not find any parameters that affect only the way that one file or file system are handled by the cache.

The main thing that I looked for was a parameter that limited the maximum size that the cache could grow to. I could not find any such parameter. This does not mean that such parameters do not exist, just that I cannot find them.

The only place that I found where you could influence the behavior of the cache was bdflush, which you had already found. Your experiments show that you cannot limit the cache growth by using bdflush.

The best place to look next is a book called "Understanding the Linux Kernel" published by O'Reilly. I read this book about 3 years ago and it has an entire chapter with a detailed description of how the kernel manages memory. It includes a detailed description of how the cache works.

http://www.amazon.com/exec/obidos/tg...glance&s=books

http://www.oreilly.com/catalog/linuxkernel2/

___________________________________
Be prepared. Create a LifeBoat CD.
http://users.rcn.com/srstites/LifeBo...home.page.html

Steve Stites